Image judgment device

ABSTRACT

The present invention provides an image judgment device that can prevent increase in a storage capacity to store element characteristic information. The image judgment device stores the element characteristic information for each element that a characteristic part of a sample object has and first and second positional information defining a position of each element, selects either the first or the second positional information, acquires image characteristic information for a partial image that is in an image frame and considered as an element specified by the first positional information in a characteristic extraction method based on a first axis when the first positional information is selected, extracts image characteristic information for a partial image that is in an image frame and considered as an element specified by the second positional information in a characteristic extraction method based on a second axis, which is acquired by rotating the first axis, when the second positional information is selected, specifies element characteristic information for an element corresponding to a position of the partial image, and judges whether or not the characteristic part appears in the image frame with use of the specified element characteristic information and the extracted image characteristic information.

TECHNICAL FIELD

The present invention relates to a technology to detect all or part ofan object such as a person or a physical object in an image frame to beshot.

BACKGROUND ART

As a conventional technology, there are technologies for detecting anobject (e.g. a person, an animal, and a physical object) or acharacteristic part (e.g. a face, and an upper body) of the object inthe image frame to be shot. As one example of the conventionaltechnologies, there is a technology for detecting a human face in animage frame to be shot by a digital still camera (hereinafter, referredto as “digital camera”),i.e., a face detection technology. The facedetection is processing to judge whether or not a face appears in anarbitrary image frame by searching within the arbitrary image framethrough certain processing. Such digital camera controls AE (AutomaticExposure)/AF (Automatic Focus) in accordance with a detected face.

Patent documents 1 and 2 disclose an image processing device and amethod for detecting a face.

For example, Patent document 1 discloses a method and an imageprocessing device for discriminating a face from an object other thanthe face (hereinafter, referred to as “non-face”). Characteristicpatterns of elements constituting a face (e.g. an eye, a nose, a mouth,and a chin) are preliminarily learned by using samples of a plurality ofhuman faces (sample object), and the discrimination is performed byusing element characteristic information composed of a plurality ofparameter values which is learned as described above.

A discriminator disclosed in Patent document 1 stores the elementcharacteristic information for each element constituting a face of thesample object in association with a placement position of each elementregarding a standard object who has a face in which each element isstandardly placed. Also, a plurality of characteristics that eachelement constituting a face can have are associated with parametervalues which are differ from each other and are included in the elementcharacteristic information for each element.

FIG. 22A shows the image processing device disclosed in Patentdocument 1. In the image processing device, an image cutout unit 33 cutsout, from an input image frame, an image used for judging whether or nota face appears (hereinafter, “judgment image”). As shown in FIG. 22B,the judgment image is cut out by scanning an image frame to be shot,gradually moving a frame 35 of appropriate pixels (e.g. 1 pixel) fromthe top left to the bottom right of the image frame to be shot in adirection from left to right or from top to bottom.

The image processing device extracts, from the cutout judgment image, apartial image based on a standard placement position of each element,and judges whether or not a face appears in the judgment image with useof (i) the element characteristic information for an element whoseposition corresponds to the placement position of the partial image and(ii) characteristics of the extracted partial image.

Patent document 1: U.S.A. Pat. No. 7,099,510, Specification

Patent document 2: Japanese Patent Application publication No.2004-62565

DISCLOSURE OF THE INVENTION The Problems the Invention is Going to Solve

Generally, since a photographer takes a picture with a digital cameraheld in a manner shown in a left drawing of FIG. 24A (normalphotographing), an image, as shown in a right drawing of FIG. 24A, inwhich a zenith direction of a person (an object) corresponds to an upperdirection of an input image is used as an input image targeted for theface detection. However, when the photographer takes a picture with thedigital camera held in a manner shown in a left drawing of FIG. 24B(vertical photographing), an image, as shown in a right drawing of FIG.24B, in which the zenith direction of a person does not correspond tothe upper direction of an input image is used as the input imagetargeted for the face detection. Hereinafter, the image in which thezenith direction of an object corresponds to the upper direction of aninput image is referred to as a normal image, whereas the image in whichthey do not correspond with each other is referred to as a rotatedimage.

Here, in order to perform the face detection for the normal image andthe rotated image by using the technology disclosed in Patent document1, the discriminator is required to prepare (store) samplecharacteristic information for each of the normal image and the rotatedimage.

The reason is that (i) a position and a direction of elements (righteye, left eye, and left side chin) constituting a face at the normalphotographing shown in FIG. 25A and (ii) a position and a direction ofthese elements constituting a face at a rotated photographing shown inFIG. 25B differ from each other, and thereby characteristics of elementsat the normal photographing and characteristics of elements at therotated photographing differ from each other. For example, the left eye(here, a rectangular area R_B1) is placed laterally at the normalphotographing, whereas the left eye is placed vertically at the rotatedphotographing, and characteristics of the lateral eye and the verticaleye differ from each other.

Generally, numerous parameter values are associated with one element.For example, the eye has various patterns of characteristics (e.g. shapeof the eye, size thereof, color thereof, a single-edged eyelid, adouble-edged eyelid, with glasses/without glasses, and the openedeye/closed eye). Therefore, parameter values for various combinations ofthese patters are required to be stored. Accordingly, as describedabove, in order to perform the face detection for the normal image andthe rotated image by using the technology disclosed in Patent document1, it is necessary to store element characteristic information for thenormal image and that for the rotated image separately. This leads to aproblem that a storage capacity of a storage area is increased.

The present invention was conceived in view of the above problem, andaims to provide an image judgment device, an image judgment method, andan integrate circuit capable of preventing increase in the storagecapacity to store the element characteristic information that is usedfor judging whether or not a characteristic part of an object appears inthe normal image or the rotated image.

Means to Solve the Problems

In order to achieve the above object, embodiment 1 of the presentinvention is an image judgment device for judging whether or not acharacteristic part of an object appears in an image frame to be shot,the characteristic part being a distinguishing aspect of the object andhaving a plurality of specific elements, the judgment being made byusing the plurality of specific elements, the image judgment devicecomprising: a first storage unit that stores element characteristicinformation regarding an erected-state sample object, the elementcharacteristic information showing characteristics of each of aplurality of specific elements that a characteristic part of the sampleobject has; a second storage unit that stores first positionalinformation and second positional information, the first positionalinformation defining a position of each of a plurality of specificelements on a specific coordinate system when a standard object is inthe same state as the erected-state sample object, the second positionalinformation defining a position of each of the specific elements on thespecific coordinate system when the standard object has been rotatedfrom the erected-state, the standard object having a characteristic parton which each of the specific elements is standardly placed; anacquisition unit operable to acquire the image frame; a selection unitoperable to select either the first positional information or the secondpositional information; an extraction processing unit operable to (i)apply the specific coordinate system to the image frame, (ii) when thefirst positional information is selected, extract a partial image thatis in the image frame and considered as an element specified by thefirst positional information, and acquire image characteristicinformation for the partial image by applying a characteristicextraction method based on a first axis on the specific coordinatesystem, and (iii) when the second positional information is selected,extract a partial image that is in the image frame and considered as anelement specified by the second positional information, and acquireimage characteristic information for the partial image by applying acharacteristic extraction method based on a second axis, the second axisbeing acquired by rotating the first axis in a direction the object isrotated; and a judgment unit operable to (i) specify the elementcharacteristic information for a specific element at a positioncorresponding to a position of the partial image on the specificcoordinate system, (ii) evaluate, for the specific element, the partialimage with use of the specified element characteristic information andthe acquired image characteristic information, and (iii) judge whetheror not the characteristic part having the specific element appears inthe image frame.

Effects of the Invention

With the above-mentioned structure, the image judgment device acquires,when the first positional information is selected, the imagecharacteristic information in the characteristic extraction method basedon the first axis, and acquires, when the second positional informationis selected, the image characteristic information in the characteristicextraction method based on the second axis. Here, the image judgmentdevice stores the element characteristic information regarding theerected-state sample object. When the first positional information isselected, the image characteristic information is acquired for a partialimage considered as the element specified by positional information whenthe standard object is in the same state as the erected-state sampleobject. Accordingly, the image judgment device can use the storedelement characteristic information and the acquired image characteristicinformation to evaluate a partial image when the object is in the samestate as the erected-state sample object. Generally, when the standardobject is rotated, a direction of specific elements that thecharacteristic part of the standard object has is changed in accordancewith a direction of rotation. Ditto a direction of the extracted partialimage and an axis being a basis to acquire the image characteristicinformation. A direction of the partial image considered as an elementspecified by the second positional information is acquired by changing adirection of the partial image considered as an element specified by thefirst positional information in accordance with a direction of rotation.Also, the second axis is acquired by rotating the first axis inaccordance with a direction of rotation. Therefore, a relativerelationship between the partial image considered as an elementspecified by the first positional information and the first axis, and arelative relationship between the partial image considered as an elementspecified by the second positional information and the second axis arethe same. Since these relative relationships are the same, the imagecharacteristic information acquired when the first positionalinformation is selected and the image characteristic informationacquired when the second positional information is selected are the samein configuration to show characteristics. Accordingly, the elementcharacteristic information stored in the first storage unit can be alsoused for the image characteristic information acquired when the secondpositional information is selected. Consequently, the image judgmentdevice does not have to separately store (i) the element characteristicinformation for an element specified by the first positional informationand (ii)) the element characteristic information for the elementspecified by the second positional information. Even when judgingwhether or not the characteristic part of the object appears in arotated image, the image judgment device can prevent increase in astorage capacity to store the element characteristic information usedfor the judgment.

Here, the acquisition unit may apply a frame surrounding thecharacteristic part of the standard object to the image frame, andacquire a judgment image surrounded by the frame from the image frame,the extraction processing unit may acquire a partial image that is inthe judgment image and considered as an element specified by selectedpositional information, and extract image characteristic information forthe partial image, and the judgment unit may (i) evaluate, for thespecific element, the partial image with use of the specified elementcharacteristic information and the extracted image characteristicinformation, and (ii) judge whether or not the characteristic partappears in the judgment image.

With the above-mentioned structure, the image judgment device can judgewhether or not the characteristic part appears in the judgment image inthe image frame.

Here, the image judgment device may further comprise: an instructionunit operable to, when the judgment unit judges that the characteristicpart does not appear in the judgment image, instruct the selection unitto select another piece of positional information which has not beenselected, wherein the selection unit selects the other piece ofpositional information which has not been selected upon receiving theinstruction, the extraction processing unit extracts another partialimage that is considered as an element specified by the selected otherpiece of positional information, and acquires image characteristicinformation for the other partial image by applying a characteristicextraction method based on an axis corresponding to the selected otherpiece of positional information, and the judgment unit (i) specifies theelement characteristic information for a specific element at a positioncorresponding to a position of the extracted other partial image on thespecific coordinate system (ii) evaluates, for the specific element, theother partial image with use of the specified element characteristicinformation and the extracted image characteristic information, and(iii) judges whether or not the characteristic part appears in the imageframe.

With the above-mentioned structure, when judging that the characteristicpart of the object does not appear in the judgment image, the imagejudgment device switches a piece of positional information to anotherpiece of positional information which has not been selected. Thisshorten the processing time in comparison with a conventional technologyin which whether or not a particular part appears is judged by rotatinga template itself or the image frame itself. The technology to judgewhether or not the particular part appears by rotating the templateitself or the image frame itself is disclosed in Patent Document 2.

Here, the extraction processing unit may comprise: an image readsub-unit that has an area in which line images of predetermined lines inthe judgment image are stored, and operable to (i) sequentially read aline image of a line from the judgment image, (ii) sequentially storethe read line image in the area, and (iii) discard, from the area, aline image having been read first when the number of the stored lineimages exceeds the number of the predetermined lines; and an extractionsub-unit operable to, each time the image read sub-unit reads and storesthe line image, (i) extract all partial images that are in the lineimages stored in the image read sub-unit and considered as elementsspecified by selected positional information, and (ii) acquire imagecharacteristic information for all the extracted partial images, and thejudgment unit may comprise: a specification sub-unit operable to, eachtime the extraction sub-unit extracts the partial image, specify elementcharacteristic information for the partial image; and a judgmentsub-unit operable to judge whether or not the characteristic partappears in the judgment image with use of the acquired imagecharacteristic information and the specified element characteristicinformation for all the acquired partial images.

With the above-mentioned structure, the image judgment device extracts,at a time when the line images are read, all partial images in thestored line images, and specifies the element characteristic informationfor each of the extracted partial images. This can prevent a processingcapacity to judge whether or not the characteristic part of the objectappears from being decreased even when the object to be detected isrotated.

The reason is described as follows.

In the conventional technology, as shown in FIG. 22A, processing orderfor the partial image is preliminarily determined. For example, theprocessing is performed with respect to the eye, nose, and mouth, inthis order. Generally, since the judgment image surrounded by the frame35 is stored in a SRAM (Static Random Access Memory), the judgment imageis required to be sequentially read from the top thereof line by line.In other wards, the partial image (a rectangular image in theconventional technology) is cut out in parallel with the reading of thejudgment image which is performed line by line, and an evaluation of anamount of characteristic for the cutout partial image is performed withuse of a parameter with respect to the partial image evaluated by acharacteristic amount evaluation unit. For example, as shown in FIG.25A, in the case where the processing is performed with respect to theeyes (a rectangular image R_A1 and R_B1), and the chin (a rectangularimage R_A2), in this order, when a line image on a1^(th) line is read,the rectangular images (R_A1 and R_B1) are selected, and an evaluationfor the rectangular images with respect to the eye is performed inparallel. Then, the rectangular image (R_A2) is selected, and anevaluation for the rectangular image with respect to the chin isperformed. On the other hand, in the case of an image shown in FIG. 25B,when performing face detection, an evaluation with respect to the eyesis performed first based on an evaluation order. In this case, when animage of one line on b1^(th) line shown in FIG. 25B is read, anevaluation for the rectangular image R_B1 with respect to the eye isperformed, and then when an image of one line on b2^(th) line is read,an evaluation for the rectangular image R_A1 with respect to the eye isperformed. Then, an evaluation for the rectangular image R_A2 withrespect to the chin is performed based on the evaluation order. That isto say, in the conventional technology, when a line image on b1^(th)line in the rotated image is read, only an evaluation for therectangular image R_B1 is performed. In other words, an evaluation forthe rectangular image R_B1 and that for the rectangular image R_A2 cannot be performed in parallel. Accordingly, the processing capacity tojudge whether or not the characteristic part appears in the rotatedimage is decreased compared with a processing capacity to judge whetheror not the characteristic part appears in a normal image. However, theimage judgment device in the present invention acquires the elementcharacteristic information for each of the extracted partial images at atime when the line image is read regardless of positional information tobe selected. The image judgment device having the above structure canperform processing using the element characteristic information for therectangular image R_B1 in FIG. 25 and processing using the elementcharacteristic information for the rectangular image R_A1 in FIG. 25 inparallel. Therefore, the image judgment device can prevent a processingcapacity to judge whether or not the characteristic part of the objectappears from being decreased even when the object to be detected isrotated.

Here, the first positional information may be shown in a first placementposition table indicating a placement position of each of the specificelements that the characteristic part of the standard object has on thespecific coordinate system when the standard object is in the same stateas the erected-state sample object, the second positional informationmay be shown in a second placement position table indicating a placementposition of each of the specific elements that the characteristic partof the standard object has on the specific coordinate system when thestandard object has been rotated from the erected-state, the secondstorage unit may further store (i) a first existence position table inwhich the placement position of each of the specific elements when thestandard object is in the same state as the erected-state sample objectis associated with an existence position of element characteristicinformation for each of the specific elements in the first storage unit,and (ii) a second existence position table in which the placementposition of each of the specific elements when the object has beenrotated from the erected-state is associated with the existence positionof the element characteristic information for each of the specificelements, the selection unit may select either a pair of the firstplacement position table and the first existence position table or apair of the second placement position table and the second existenceposition table, the extraction sub-unit may extract the partial imagewith use of a placement position table included in the pair selected bythe selection unit, and the specification sub-unit may specify theelement characteristic information for the partial image with use ofpositional information having been used to extract the partial image andan existence position table selected by the selection unit.

With the above-mentioned structure, by using the pair of the firstplacement position table and the first existence position table and thepair of the second placement position table and the second existenceposition table stored in the second storage unit, the image judgmentdevice can manage the first and second positional information, associatethe first positional information with the element characteristicinformation, and associate the second positional information with theelement characteristic information.

Here, shape information showing a shape of a partial image to beacquired may be associated with the placement position of each of thespecific elements included in each of the first placement position tableand the second placement position table, the extraction sub-unit mayextract a partial image that is at the placement position of each of thespecific elements from the stored line images, the partial image havinga shape shown by the shape information corresponding to the placementposition.

With the above-mentioned structure, the image judgment device canspecify the shape of the partial image to be extracted.

Here, the element characteristic information may be an evaluation valuegroup composed of evaluation values having been weighted in terms of anapproximation to a corresponding specific element, each amount ofcharacteristic shown by a combination of brightness and darkness in aplurality of areas constituting the partial image may be associated witha different sample value, the existence position included in theexistence position table may show a starting position of a correspondingsample value group in the first storage unit, the extraction sub-unitmay acquire, as the image characteristic information, the amount ofcharacteristic of the partial image by applying a characteristicextraction method based on an axis corresponding to the selectedpositional information, the specification sub-unit may specify, eachtime the extraction sub-unit extracts the partial image, a startingposition of the evaluation value group with use of the selectedexistence position table and the placement position having been used toextract the partial image, the judgment sub-unit may (i) acquire anevaluation value corresponding to the amount of characteristic of thepartial image from the evaluation value group shown by the specifiedstarting position, (ii) calculate a weighted value in terms of anapproximation to the characteristic part from a total of the acquiredevaluation values for all the partial images in the judgment image, and(iii) judge that the characteristic part appears in the judgment imagewhen the calculated weighted value is a predetermined threshold value ormore.

With the above-mentioned structure, the image judgment device canspecify the starting position of the evaluation value group for each ofthe extracted partial images with use of the existence position tableselected by the selection unit. And it can acquire, for each of theextracted partial image, the evaluation value corresponding to theamount of characteristic for the acquired partial image from theevaluation value group shown by the specified starting position. Furtheralso the image judgment device can calculate a weighted value in termsof an approximation to the characteristic part with use of a total ofthe acquired evaluation values for all the partial images extracted fromthe judgment image, and judge whether or not the characteristic partappears in the judgment image based on the calculated weighted value.

Here, the second storage unit may further store, (i) in association withthe first placement position table and the first existence positiontable, a first placement position table use flag and a first existenceposition table use flag, respectively, showing whether or not each ofthe tables can be used and, (ii) in association with the secondplacement position table and the second existence position table, asecond placement position table use flag and a second existence positiontable use flag, respectively, showing whether or not each of the tablescan be used, the selection unit may further set a value for a placementposition table use flag and an existence position table use flagcorresponding to a pair of the placement position table and theexistence position table to be selected, the value showing that thecorresponding placement position table and existence position table arepermitted to be used, the extraction sub-unit may use the placementposition table for which the value showing that the placement positiontable is permitted to be used is set to extract the partial image, andthe specification sub-unit may use the existence position table forwhich the value showing that the existence position table is permittedto be used is set to specify the element characteristic information.

Also, the second storage unit may have a referable area and anon-referable area, the selection unit may store a pair of the placementposition table and the existence position table to be selected in thereferable area, and the other pairs of the placement position table andthe existence position table in the non-referable area, the extractionsub-unit may use the placement position table stored in the referablearea to extract the partial image, and the specification sub-unit mayuse the existence position table stored in the referable area to specifythe element characteristic information.

Further also, an access right may be associated with each of the firstplacement position table, the second placement position table, the firstexistence position table, and the second existence position table, theaccess right showing whether or not each of the tables can be accessed,the selection unit may set (i) an access right for a pair of theplacement position table and the existence position table to beselected, the access right showing that the corresponding placementposition table and existence position table are permitted to beaccessed, and (ii) an access right for pairs of the other placementposition tables and existence position tables, the access right showingthat the corresponding placement position tables and existence positiontables are not permitted to be accessed, the extraction sub-unit may usethe placement position table for which the access right showing theplacement position table is permitted to be accessed is set to extractthe partial image, and the specification sub-unit may use the existenceposition table for which the access right showing the existence positiontable is permitted to be accessed is set to specify the elementcharacteristic information.

With the above-mentioned structure, the image judgment device can easilyswitch pairs of the placement position table and the existence positiontable to be selected with use of the use flag for each table, thereferable area and the non-referable area, and the access right for eachtable.

Here, the selection unit may receive an instruction to select either thefirst or second positional information from the outside, and selectpositional information shown by the received instruction.

With the above-mentioned structure, the image judgment device can easilyselect either the first positional information or the second positionalinformation by receiving the instruction from the outside.

Here, the object maybe rotated 90 degrees, 180 degrees, 270 degrees, orreversed.

With the above-mentioned structure, since the image judgment devicestores the second positional information corresponding to one of therotations of 90 degrees, 180 degrees, 270 degrees, and reverse, there isno need to rotate the template itself or the image frame itself 90degrees, 180 degrees, 270 degrees, or reverse as disclosed in theconventional technology. This shorten the processing time in comparisonwith the conventional technology in which whether or not a particularpart appears is judged by rotating the template itself or the imageframe itself.

Here, the object may be a person, and the characteristic part may be aface.

With the above-mentioned structure, the image judgment device canprevent increase in a storage capacity to store the elementcharacteristic information used to judge whether or not a human faceappears in the image frame to be shot regardless of rotation.

Here, the image judgment device may be included in an imaging deviceshooting the object.

With the above-mentioned structure, since the image judgment device isincluded in the imaging device, the imaging device can judge whether ornot the characteristic part of the object appears when shooting theobject.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a block diagram showing a structure of an image processingdevice 1.

FIG. 2 schematically shows an image stored in an image memory 4.

FIG. 3 shows a structure of selector circuits 11 a and 11 b.

FIG. 4A shows rectangular image areas to be selected at a standard angle(0 degrees). FIGS. 4B, C, and D show rectangular image areas to beselected at rotating angles (90 degrees, reverse, and 270 degrees,respectively).

FIG. 5A shows a coordinate of a rectangular image area to be selected ata standard angle (0 degrees). FIGS. 5B, C, and D show coordinates ofrectangular image areas to be selected at rotating angles (90 degrees,reverse, and 270 degrees, respectively). FIG. 5E shows a relationshipamong coordinates of start positions of rectangular image areas to beselected at respective angles.

FIG. 6 shows an example of a data structure of a selector table T11 a ata rotating angle 0 degrees.

FIG. 7 shows an example of a data structure of a selector table T11 b ata rotating angle 90 degrees.

FIG. 8 shows an example of a data structure of a selector table T11 c ata reverse angle.

FIG. 9 shows an example of a data structure of a selector table T11 d ata rotating angle 270 degrees.

FIG. 10 shows, when separating a rectangular image into two areas,ranges of values in the separated areas calculated to acquire amounts ofcharacteristic at respective angles.

FIG. 11 shows, when separating a rectangular image into nine areas,ranges of values in the separated areas calculated to acquire amounts ofcharacteristic at respective angles.

FIG. 12 shows a memory map for rectangle information A stored in a ROM9.

FIG. 13 shows a memory map for rectangle information B stored in the ROM9.

FIG. 14A shows an example of a data structure of an address conversiontable T31 a at a rotating angle 0 degrees. FIG. 14B shows an example ofa data structure of an address conversion table T31 b at a rotatingangle 90 degrees.

FIG. 15A shows an example of a data structure of an address conversiontable T31 c at a reverse angle. FIG. 15B shows an example of a datastructure of an address conversion table T31 d at a rotating angle 270degrees.

FIGS. 16A, B, C, and D show relationships between angles (0 degrees, 90degrees, reverse, and 270 degrees) set by an angle set unit 10 andangular ranges to detect a face.

FIG. 17 is a flow chart showing operation of switching processing.

FIG. 18 is a flow chart showing operation of face detection processing.

FIG. 19 is a block diagram showing a structure of an image processingdevice 1 a.

FIG. 20 is a block diagram showing a structure of a semiconductorintegrated circuit 70.

FIG. 21 is a block diagram showing a structure of an imaging device 100.

FIG. 22A is a reference drawing showing a conventional image processingdevice. FIG. 22B is a reference drawing showing an order of scanning byusing a search window pertaining to face detection.

FIGS. 23A, B, C, D, and E show examples of rectangular areas to beselected.

FIG. 24A shows a relationship between a zenith direction of a person andthat of an input image at a normal photographing with a digital camera.FIG. 24B shows a relationship between a zenith direction of a person andthat of an input image at a vertical photographing with the digitalcamera.

FIG. 25A shows positions of rectangular images to be selected at thenormal photographing. FIG. 25B shows positions of rectangular images tobe selected at the vertical photographing.

DESCRIPTION OF CHARACTERS

1 image processing device

2 image input unit

3 image acquisition unit

4 image memory

5 face detection unit

6 rectangular image cutout unit

7 classifier

8 ROM address calculation unit

9 ROM

10 angle set unit

11 cutout processing unit

11 a, 11 b selector circuit

12 selector table storage unit

13 characteristic amount calculation unit

14 evaluation value calculation unit

15 judgment unit

16 address conversion table storage unit

17 address generation unit

18 counter

19 processing control unit

BEST MODE FOR CARRYING OUT THE INVENTION 1. Embodiment 1

The following describes embodiment 1 of the present invention withreference to the drawings.

1. 1 Structure of an Image Processing Device 1

FIG. 1 is a block diagram showing an image processing device 1 inembodiment 1.

As shown in FIG. 1, the image processing device 1 includes an imageinput unit 2, a face detection unit 5, and an angle set unit 10. Theimage processing device 1 is an image judgment device for judgingwhether or not a human face appears in an input image frame.

Specifically, the image processing device 1 is a computer system that iscomposed of a micro processor, a ROM, a RAM, a hard disc unit, and thelike. A computer program is stored in the RAM or the hard disc unit. Asthe micro processor operates in accordance with the computer program,the image processing device 1 achieves its functions. Here, the computerprogram is constituted from a plurality of instruction codes combinedwith each other to achieve a predetermined function, the instructioncodes showing commands for a computer.

(1) Image Input Unit 2

As shown in FIG. 1, the image input unit 2 includes an image acquisitionunit 3 and an image memory 4.

The image acquisition unit 3 includes an interface circuit for acquiringimage data (input image data) to be shot and so on.

The image memory 4 stores the acquired image data. Specifically, theimage memory 4 is composed of a SRAM, and has a 24 word structure inwhich pixel data of one line (i.e. data of 24 pixels, here) is stored inone word. In other words, image data acquired by the image acquisitionunit 3 is once stored in the image memory 4 in a unit in which the facedetection unit 5 performs processing, i.e., 24 pixels×24 pixels as shownin FIG. 2. Hereinafter, the unit in which the face detection unit 5performs processing (e.g. 24 pixels×24 pixels) is referred to as awindow. Pixel data of one window is stored in the image memory 4.

Based on a size of a window, the image acquisition unit 3 scans an inputimage to be shot by gradually moving appropriate pixels (e.g. 1 pixel)from the top left to the bottom right of the input image in a directionfrom left to right or from top to bottom, and acquires pixel data of awindow to store in the image memory 4.

The pixel data stored in the image memory 4 is output to the facedetection unit 5 in units of line (here, pixel data of 24 pixels) inresponse to a request from the face detection unit 5.

Here, the image memory 4 needs to have only a capacity of one window.However, the capacity is not limited to this. The image memory 4 mayhave a capacity larger than one window to store pixel data acquired bythe image acquisition unit 3 and output pixel data to the face detectionunit 5 in parallel so that a delay in acquisition of the pixel data canbe concealed.

Note that a size of a window, i.e., 24 pixels×24 pixels here and astructure of the SRAM are just examples, and not limited to these. Awindow of another size may be used, and the image memory 4 may be astorage unit other than the SRAM.

(2) Face Detection Unit 5

As shown in FIG. 1, the face detection unit 5 includes a counter 18, arectangular image cutout unit 6, a classifier 7, a ROM addresscalculation unit 8, and a ROM 9.

(2-1) Counter 18

The counter 18 counts up from 0 to 23 and specifies a line in the imagememory 4 to be read according to a counter value.

The counter value of the counter 18 is output to the image memory 4 asan address to read out all pieces of pixel data of one window stored inthe image memory 4. At this time, image data of 24 pixels on a linewhose number is the same as a value shown by the counter 18 is outputfrom the image memory 4 to the face detection unit 5. For example, whena value of the counter 18 is 15, pixel data of 24 pixels on 15^(th) lineis output to the face detection unit 5.

The counter value of the counter 18 is also output to the rectangularimage cutout unit 6 as a line number in a window to be read in.

(2-2) Rectangular Image Cutout Unit 6

As shown in FIG. 1, the rectangular image cutout unit 6 includes acutout processing unit 11 and a selector table storage unit 12. Therectangular image cutout unit 6 cuts out a rectangular image from imagedata of a window output from the image input unit 2.

The selector table storage unit 12 is a storage area that storesinformation about a cutout position of a rectangular image.

The cutout processing unit 11 selects (cuts out) image data of arectangular image (hereinafter, also referred to as “rectangular imagedata”) based on one of selector tables.

(2-2-1) Cutout Processing Unit 11

As shown in FIG. 1, the cutout processing unit 11 includes selectorcircuits 11 a and 11 b.

The selector circuits 11 a and 11 b read out, from the image memory 4,pixel data of one line in a window on a line whose number is the same asa counter value of the counter 18. Here, all pieces of data in thewindow are input to the rectangular image cutout unit 6 through 24cycles.

The selector circuits 11 a and 11 b select a rectangular image at apredetermined position based on the information stored in the selectortable storage unit 12.

A circuit diagram of the selector circuit 11 a (11 b) is shown in FIG.3. The selector circuit 11 a includes a register 21 for storing data ofone pixel, a shift register group 20 being composed of a plurality ofthe registers 21 (here, 24 registers), and a column selector circuit 22.

The selector circuit 11 a needs to store pixel data of a rectangularimage, because the classifier 7 (described later) performs processing ofone rectangular image at a time. A buffer to store the pixel data of arectangular image is composed of the shift register group 20. Byincluding four tiers of the shift register group 20 in FIG. 3, arectangular image with a height of 4 pixels can be processed. The pixeldata of a rectangular image stored in the shift register group 20 isselected by the column selector circuit 22 based on the informationstored by the selector table storage unit 12, and output to theclassifier 7.

Note that explanations about the selector circuit 11 b are omitted here,because the structure of the selector circuit 11 b is the same as thatof the selector circuit 11 a.

The selector circuits 11 a and 11 b can read out pieces of pixel data onthe same line whose number is the same as a value shown by the counter18 at one time. When there are two rectangular images on the same line,two pieces of rectangular image data can be selected at one time byusing both of the selector circuits 11 a and 11 b.

(2-2-2) Selector Table Storage Unit 12

The selector table storage unit 12 stores information showing a positionof rectangular image data to be cut out by the selector circuits 11 aand 11 b. Here, suppose that 4 pixels×4 pixels rectangular images B1,B2, and B3 shown in FIGS. 4A to 4D are to be selected (cutout). In thefollowing, explanations are made by using template frames shown in FIGS.4A to 4D and rectangular images.

The selector table storage unit 12 stores information about startpositions B1 a, B2 a, and B3 a of rectangular images B1, B2, and B3,respectively, to be selected based on a template frame B10 shown in FIG.4A. A size of the template frame B10 is 24 pixels vertical×24 pixelshorizontal, and it is the same as a size of a window stored in the imagememory 4. Note that, in a case where a human face appears in thetemplate frame B10, rectangular images B1, B2, and B3 are placed onstandard placement positions of the right eye, left eye, and left sidechin, respectively.

Also, the selector table storage unit 12 stores start positions ofrectangular images B1, B2, and B3, respectively, when the template frameB10 shown in FIG. 4A is rotated 90 degrees and 270 degrees in acounterclockwise direction. Here, the selector table storage unit 12stores start positions B1 b, B2 b, and B3 b shown in FIG. 4B and startpositions B1 d, B2 d, and B3 d shown in FIG. 4D. Further also, theselector table storage unit 12 stores start positions of rectangularimages B1, B2, and B3, respectively, when the template frame B10 shownin FIG. 4A is reversed. Here, the selector table storage unit 12 storesstart positions B1 c, B2 c, and B1 c shown in FIG. 4C.

FIG. 5 shows a relationship between a rectangular image and a rotatingangle. Here, suppose that an X-coordinate and a Y-coordinate of a startposition of a rectangular image B4 a with h pixels×w pixels in atemplate frame B11 at a standard angle (0 degrees) is shown as (x, y).In this case, when the template B11 is rotated 90 degrees in acounterclockwise direction, the rectangular image B4 a is moved to aposition shown by a rectangular image B4 b, and an X-coordinate of itsstart position is y and a Y-coordinate thereof is 24-x-w as shown inFIG. 5B. When the template frame B11 is reversed, or rotated 270 degreesin a counterclockwise direction, the rectangular image B4 a is moved topositions shown by rectangular images B4 c or B4 d, respectively, asshown in FIGS. 5C and 5D. A relationship between a coordinate positionat a standard angle and coordinate positions after being rotated orreversed is shown in FIG. 5E.

The following describes a specific example of how information about arectangular image to be selected is stored in the selector table storageunit 12.

The selector table storage unit 12 stores four kinds of selector tablesT11 a, T11 b, T11 c, and T11 d shown in FIGS. 6 to 9.

Each of the selector tables T11 a, T11 b, T11 c, and T11 d is composedof an area which includes at least one combination of a window linenumber, rectangle information A, and rectangle information B.

The window line number is a value showing a line number in a window.

Each of the rectangle information A and the rectangle information B iscomposed of information about whether or not a rectangular image isincluded, and an X-coordinate of the rectangular image.

Specifically, the information about whether or not a rectangular imageis included is a rectangular image flag which is set to a value 0 or 1.When the value of the rectangular image flag is set to 0, there is norectangular image whose Y-coordinate of a start position is the same asa corresponding line number. In contrast, when the value of therectangular image flag is set to 1, there is a rectangular image whoseY-coordinate of a start position is the same as a corresponding linenumber.

The X-coordinate of the rectangular image shows an X-coordinate of astart position of an existing rectangular image, when the informationabout whether or not a rectangular image is included, that is to say, avalue of the rectangular image flag is set to 1.

Here, an X-coordinate and a Y-coordinate of a start position of theexisting rectangular image is specified by the window line number andthe X-coordinate of the rectangular image.

The selector table T11 a shown in FIG. 6 shows information pertaining torectangular images in FIG. 4A, that is, at a standard angle (0 degrees).It shows that there are two rectangular images corresponding to a windowline number “y_b” and one rectangular image corresponding to a windowline number “y_c”. In this case, a rectangular image B1 shown in FIG. 4Ais specified by the window line number “y_b” and an X-coordinate of arectangular image “x_b”, and a rectangular image B2 shown in FIG. 4A isspecified by the window line number “y_b” and an X-coordinate of arectangular image “x_b′”. Also, a rectangular image B3 shown in FIG. 4Ais specified by the window line number “y_c” and the X-coordinate of arectangular image “x_b′”.

Each of the selector tables T11 b, T11 c, and T11 d shown in FIGS. 7 to9 shows information pertaining to rectangular images in FIGS. 4B to 4D,respectively.

In the selector table T11 b, two rectangular images specified by awindow line number “24-x_b′-4” show rectangular images B2 and B3 in FIG.4B. Here, the rectangular image B3 is shown by the rectangle informationA, and the rectangular image B2 is shown by the rectangle information B.Also, a rectangular image specifie d by a window line number “24-x_b-4”shows a rectangular image B1 in FIG. 4B.

In the selector table T11 c, a rectangular image specified by a windowline number “24-y_c-4” shows a rectangular image B3 in FIG. 4C, and tworectangular images specified by a window line number “24-y_b-4” showrectangular images B1 and B2 in FIG. 4C. Here, the rectangular image B1is shown by the rectangle information A, and the rectangular image B2 isshown by the rectangle information B.

In the selector table T11 d, a rectangular image specified by a windowline number “x_b” shows a rectangular image B1 in FIG. 4D, and tworectangular images specified by a window line number “x_b′ ” showrectangular images 32 and B3 in FIG. 4D. Here, the rectangular image B3is shown by the rectangle information A, and the rectangular image B2 isshown by the rectangle information B.

Each of the selector tables T11 a, T11 b, T11 c, and T11 d is associatedwith a selector table use flag showing whether or not each of theselector tables can be used by the selector circuits 11 a and 11 b. Whena value of the selector table use flag is set to 0, the correspondingselector table cannot be used, whereas the value is set to 1, it can beused. Here, a value of the selector table use flag corresponding to oneof the four selector tables T11 a, T11 b, T11 c, and T11 d is set to“1”, and values of the selector table use flags corresponding to theother selector tables are set to “0”. In other words, when selectingrectangular image data, the selector circuits 11 a and 11 b refer toonly one selector table at all times.

In this embodiment, the selector circuit 11 a is described to selectrectangular image data shown by the window line number and the rectangleinformation A, and the selector circuit 11 b is described to selectrectangular image data shown by the window line number and the rectangleinformation B.

(2-2-3) Specific Example of Cutting Out of Rectangular Image Data

Here, a specific example of cutting out of rectangular image data isdescribed with use of the selector table T11 a in FIG. 6.

The selector table T11 a uses a value of the counter 18 corresponding toa line number of pixel data in a window input to the selector circuits11 a and 11 b and outputs information about whether or not a rectangularimage is included on each line in a window and positional information ofthe rectangular image in a column direction (X direction). The columnselector circuit 22 included in the selector circuit 11 a selectsrectangular image data based on these pieces of information. Informationstored in the selector table T11 a when cutting out a rectangular imageB1 in FIG. 4A is shown as rectangular image information T12 a in FIG. 6.A start position of the rectangular image B1 in FIG. 4A is shown by acoordinate in a window (x, y)=(x_b, y_b), so the rectangular image B1 isshown by the rectangular image information T12 a in FIG. 6. When theinformation is input to the selector circuit 11 a, the information aboutwhether or not a rectangular image is included corresponding to y_b^(th)line shows “1” (i.e. a rectangular image is included), and a

X-coordinate of the rectangular image shows x_b. Accordingly, when dataon y_b^(th) line is shifted to n^(th) line in the shift register group,the column selector circuit 22 included in the selector circuit 11 aselects pixel data Rect (x_b) in column x_b (pixel data of 16 pixelsshown by a frame T1) to output to the classifier 7.

At this time, the selector circuit 11 b selects rectangular image dataof a rectangular image B2 in FIG. 4A, and outputs the selectedrectangular image data to the classifier 7.

(2-3) Classifier 7

As shown in FIG. 1, the classifier 7 includes a characteristic amountcalculation unit 13, an evaluation value calculation unit 14, and ajudgment unit 15. The classifier 7 calculates an amount ofcharacteristic and an evaluation value of each rectangular image withuse of a plurality of pieces of rectangular image data output from therectangular image cutout unit 6, and identifies whether or not a faceappears in a window based on the evaluation value.

(2-3-1) Characteristic Amount Calculation Unit 13

The characteristic amount calculation unit 13 calculates an amount ofcharacteristic for each of one or more rectangular images selected bythe selector circuits 11 a and 11 b.

The characteristic amount calculation unit 13 outputs the calculatedamount of characteristic to the ROM address calculation unit 8.

The following describes a calculation method disclosed in PatentDocument 1 as an example of how to calculate an amount ofcharacteristic.

For example, an amount of characteristic of a rectangular image shown inFIG. 23B is calculated by using a difference between a sum of pixels ina black rectangular area with 4 pixels horizontal×2 pixels vertical inan upper half of the rectangular image and a sum of pixels in a whiterectangular area with 4 pixels horizontal×2 pixels vertical in a lowerhalf of the rectangular image. First, an amount of characteristic of theupper rectangular area is calculated, and then that of the lowerrectangular area is calculated. And the amount of characteristic of therectangular image is calculated by setting characteristic of arectangular area with larger sum of pixels (brighter) as “1”, settingcharacteristic of a rectangular area with smaller sum of pixels (darker)as “0”, and arranging the above characteristics of the black and whiterectangular areas included in the rectangular image as binaryinformation. In FIG. 23B, when the black rectangular area is darker andthe white rectangular area is brighter, “01” is acquired by arrangingcharacteristics from the characteristic of the upper area to that of thelower area. The binary value “01” acquired at this time is the amount ofcharacteristic. Similarly, in FIG. 23C, an amount of characteristic“101” is acquired by arranging characteristics from the characteristicof the left area.

In this embodiment, as shown in FIG. 23B, the amount of characteristicis calculated by using a difference between the sum of pixels in theupper rectangular area with 4 pixels horizontal×2 pixels vertical andthe sum of pixels in the lower rectangular area with 4 pixelshorizontal×2 pixels vertical based on a rectangular area to be selectedat a standard angle (0 degrees). FIGS. 10A to 10D show ranges of valuesof the amount of characteristic at rotating angles (0 degrees, 90degrees, reverse, and 270 degrees), respectively, in this embodiment.FIG. 10A shows a range of values of the amount of characteristic at astandard angle (0 degrees). In this case, the characteristic amountcalculation unit 13 separates a rectangular image into upper and lowerareas by using a horizontal axis B103 a, and an amount of characteristic“AB” is acquired by arranging a sum of pixels “A” in a first area B101 aand a sum of pixels “B” in a second area B102 a in this order (from topto bottom).

FIG. 10B shows a range of values when the rectangular image is rotated90 degrees in a counterclockwise direction. In this case, a positionrelation between the first area 101 a and the second area 102 a, whichare arranged vertically at a rotating angle 0 degrees, is changed. Thefirst area 101 a and the second area 102 a are arranged horizontally.Accordingly, the characteristic amount calculation unit 13 separates arectangular image after being rotated 90 degrees by using a verticalaxis B103 b, which is acquired by rotating the axis B103 a 90 degrees ina counterclockwise direction, and an amount of characteristic “AB” isacquired by arranging a sum of pixels “A” in a first area B101 b and asum of pixels “B” in a second area B102 b in this order (from left toright).

FIG. 10C shows a range of values when the rectangular image is reversed.In this case, the rectangular image is separated in the same manner asthat separated at a rotating angle 0 degrees. However, it is separatedby using an axis B103 c. At this time, a position relation between afirst area 101 c and a second area 102 c after the rectangular image isseparated becomes opposite to the position relation at a rotating angle0 degrees. Accordingly, the characteristic amount calculation unit 13calculates an amount of characteristic “AB” by arranging a sum of pixels“A” in a first area B101 c and a sum of pixels “B” in a second area B102c in this order (from bottom to top).

FIG. 10D shows a range of values when the rectangular image is rotated270 degrees in a counterclockwise direction. In this case, therectangular image is separated in the same manner as that separated at arotating angle 90 degrees. However, it is separated by using an axisB103 d. At this time, a position relation between a first area 101 d anda second area 102 d after the rectangular image is separated becomesopposite to the position relation at a rotating angle 90 degrees.Accordingly, the characteristic amount calculation unit 13 calculates anamount of characteristic “AB” by arranging a sum of pixels “A” in afirst area B101 d and “B” in a second area B102 d in this order (fromright to left).

As described above, it can be seen that a range of values of an amountof characteristic is specified in accordance with a direction of an axisused to separate a rectangular image.

According to the calculation method, amounts of characteristic of arectangular image are the same regardless of rotating angle (0 degrees,90 degrees, reverse, and 270 degrees).

Note that the calculation method to calculate an amount ofcharacteristics of a rectangular image is an example, and not limited tothis.

Also, in this embodiment, an amount of characteristic of a rectangularimage is calculated by separating the rectangular image into two areas.However, a method is not limited to this.

The rectangular image may be separated in a different method.

FIGS. 11A to 11D shows, as the different method, ranges of a value of anamount of characteristic of a rectangular image with 9 pixels×9 pixelswhen the rectangular image is separated into nine areas.

FIG. 11A shows a range when the rectangular image is rotated 0 degrees.In this case, an amount of characteristic is acquired by arrangingvalues from a value of the top left area in the rectangular image in adirection from left to right and from top to bottom. That is to say,“ABCDEFGHI” is acquired as the amount of characteristic.

FIG. 11B shows a range when the rectangular image is rotated 90 degrees.In this case, an amount of characteristic is acquired by arrangingvalues from a value of the bottom left area in the rectangular image ina direction from bottom to top and from left to right. That is to say,“ABCDEFGHI”, which is the same as the amount of characteristic acquiredat 0 degrees, is acquired as the amount of characteristic.

FIG. 11C shows a range when the rectangular image is reversed. In thiscase, an amount of characteristic is acquired by arranging values from avalue of the bottom left area in the rectangular image in a directionfrom left to right and from bottom to top. That is to say, “ABCDEFGHI”,which is the same as the amount of characteristic acquired at 0 degrees,is acquired as the amount of characteristic.

FIG. 11D shows a range when the rectangular image is rotated 270degrees. In this case, an amount of characteristic is acquired byarranging values from a value of the top right area in the rectangularimage in a direction from top to bottom and from right to left. That isto say, “ABCDEFGHI”, which is the same as the amount of characteristicacquired at 0 degrees, is acquired as the amount of characteristic.

Therefore, the amounts of characteristic of the rectangular image showthe same value regardless of angle.

When a rectangular image is separated into nine areas, a plurality ofaxises are used to separate the rectangular image. By using one of theseaxises, a range of an amount of characteristic is specified. Forexample, when a horizontal right pointing axis (e.g. an axis B104 a) isused at 0 degrees, a range of an amount of characteristic of arectangular image at the angle is specified. When the axis is rotatedaccording to a rotating angle, a range of an amount of characteristic ofa rectangular image at the rotating angle is specified by using an axisafter being rotated (an axis B104 b, B104 c, and B104 d).

(2-3-2) Evaluation Value Calculation Unit 14

The evaluation value calculation unit 14 calculates an evaluation valuefor a window with use of the amount of characteristic and a learningparameter (described later). Here, the learning parameter is a weightedvalue for the amount of characteristic. In order to use the learningparameter to calculate the evaluation value for a window, the learningparameter is associated with each amount of characteristic and acquiredby preliminarily learning a large number of sample images.

The evaluation value calculation unit 14 receives, from the ROM addresscalculation unit 8 (described later), a learning parameter which isassociated with a rectangular image selected by the selector circuit.

Upon receiving learning parameters corresponding to all rectangularimages selected from one window, the evaluation value calculation unit14 calculates accumulation values thereof, and the calculation resultbecomes an evaluation value.

The evaluation value calculation unit 14 outputs the calculatedevaluation value to the judgment unit 15.

(2-3-3) Judgment Unit 15

The judgment unit 15 judges whether or not a face appears in a windowbased on the evaluation value of one window.

Specifically, the judgment unit 15 judges whether or not a face appearsby comparing the evaluation value and a threshold value which ispreliminarily stored in the judgment unit 15.

When the evaluation value is the threshold value or larger, the judgmentunit 15 judges a target of the judgment as a face (i.e. a face appearsin a window), when the evaluation value is smaller than the thresholdvalue, the judgment unit 15 judges it as a non-face (i.e. a face doesnot appears in a window). The judgment result is output to the outsidefrom the classifier 7.

Note that the calculation method of the evaluation value and thejudgment method of a face/non-face are just examples, and not limited tothese.

(2-4) ROM 9

The ROM 9 is a learning parameter storing unit for storing the learningparameter used by the evaluation value calculation unit 14 to calculatethe evaluation value.

Examples of memory maps T20 and T21 of the learning parameter stored inthe ROM 9 are shown in FIGS. 12 and 13.

In the memory map T20, for each of one or more rectangular images shownin a column of the rectangle information A in a selector table, learningparameters for amounts of characteristic which the rectangular image canhave are stored.

In the memory map T21, for each of one or more rectangular images shownin a column of the rectangle information B in a selector table, learningparameters for amounts of characteristic that the rectangular image canhave are stored.

For example, four learning parameters from an offset address “offset An”in FIG. 12 are learning parameters for amounts of characteristic thatthe rectangular image B1 shown in FIG. 4 can have. Also, four learningparameters from an offset address “offset Am” in FIG. 12 are learningparameters for amounts of characteristic that the rectangular image B3shown in FIG. 4 can have. Further also, four learning parameters from anoffset address “offset En” in FIG. 13 are learning parameters foramounts of characteristic that the rectangular image B2 shown in FIG. 4can have.

Note that a method for specifying a learning parameter corresponding toa rectangular image is described later.

(2-5) ROM Address Calculation Unit 8

As shown in FIG. 1, the ROM address calculation unit 8 includes anaddress conversion table storage unit 16 and an address generation unit17. The ROM address calculation unit 8 is a learning parameter referencecalculation unit for calculating a reference address in the ROM 9 withuse of the amount of characteristic output from the classifier 7.

(2-5-1) Address Conversion Table Storage Unit 16

The address conversion table storage unit 16 stores an addressconversion table composed of conversion information to convert thereference address in the ROM 9.

The address conversion table storage unit 16 stores address conversiontables T31 a, T31 b, T31 c and T31 d for the template frames shown inFIGS. 4A to 4D, respectively. Each of the address conversion tablesshows an offset address of a storage position of a learning parameterwhich is stored in the ROM 9 and to be referred by a rectangular imageselected based on one of the template frames. Examples of the addressconversion tables T31 a, T31 b, T31 c, and T31 d are shown in FIGS. 14Aand 14B, and FIGS. 15A and 15B. The address conversion table T31 a showsan offset address of a learning parameter which is stored in the ROM 9and to be referred by each rectangular image shown in FIG. 4A. Theaddress conversion table T31 b shows an offset address of a learningparameter which is stored in the ROM 9 and to be referred by eachrectangular image shown in FIG. 4B. The address conversion table T31 cshows an offset address of a learning parameter which is stored in theROM 9 and to be referred by each rectangular image shown in FIG. 4C.And, the address conversion table T31 d shows an offset address of alearning parameter which is stored in the ROM 9 and to be referred byeach rectangular image shown in FIG. 4D. This means that the addressconversion tables T31 a, T31 b, T31 c, and T31 d correspond to theselector tables T11 a, T11 c, and T11 d, respectively.

Each of the address conversion tables T31 a to T31 d has a storage areaincluding one or more combinations of a window line number, output(offset) for the rectangle information A, and output (offset) for therectangle information B.

The window line number is a value showing a line number in a window.

The output (offset) for the rectangle information A shows an offset,when a value shown by the window line number is a Y-coordinate of astart position, to be referred by a rectangular image specified by therectangle information A in a corresponding selector table.

The output (offset) for the rectangle information B shows an offset,when a value shown by the window line number is a Y-coordinate of astart position, to be referred by a rectangular image specified by therectangle information B in a corresponding selector table.

Each of the address conversion tables T31 a, T31 b, T31 c, and T31 d isassociated with an address conversion table use flag showing whether ornot each address conversion table can be used by the address generationunit 17 (described later). When a value of the address conversion tableuse flag is set to 0, the corresponding address conversion table cannotbe used, whereas a value of the address conversion table use flag is setto 1, it can be used. Here, a value of the address conversion table useflag corresponding to one of the four address conversion tables T31 a,T31 b, T31 c, and T31 d is set to “1”, and values of the addressconversion table use flags corresponding to the other address conversiontables are set to “0”. In other words, the address generation unit 17refers to only one address conversion table at all times.

(2-5-2) Address Generation Unit 17

The address generation unit 17 generates (calculates) a referenceaddress (ROM address) of a learning parameter with use of an amount ofcharacteristic output from the classifier 7 in accordance with a usableaddress conversion table.

Upon receiving the amount of characteristic from the characteristicamount calculation unit 13, the address generation unit 17 calculates aROM address showing an address in the ROM 9 to be accessed with use ofthe usable address conversion table and the following equation 1.

ROM address=Table (window line number)+an amount of characteristic  (Equation 1)

Table (window line number) in the equation 1 is an offset addressacquired from the usable address conversion table. Also, the amount ofcharacteristic is an amount of characteristic input from the classifier7, and it is acquired by arranging the brightness and darknessinformation.

As an example, the window line number is used to acquire the offsetaddress from the Table, but it is not limited. For example, the order ofa rectangular image which has been preliminarily determined throughlearning may be used.

The address generation unit 17 accesses the ROM 9 based on thecalculated ROM address, and acquires a learning parameter correspondingto an amount of characteristic which is calculated for a rectangularimage in process. The acquired learning parameter is output to theevaluation value calculation unit 14 included in the classifier 7.

The following describes a case where the rectangular image B1 shown inFIG. 4 has the same characteristics as those of a rectangular imageshown in FIG. 23B, that is, a case where an amount of characteristic is“01”. In this case, the rectangular image B1 is specified by therectangle information A in the selector table T11 a shown in FIG. 6.Therefore, the address generation unit 17 acquires an offset address“offset An” first by using a window line number y_b in the addressconversion table T31 a. Then, since the amount of characteristic of therectangular image B1 is “01”, the address generation unit 17 calculatesa ROM address “offset An+1” with use of the equation 1 and the acquiredoffset address “offset An”. Here, it can be found that the learningparameter corresponding to the rectangular image B1 is stored in anaddress shown by the calculated ROM address “offset An+1”. Based on thecalculated ROM address “offset An+1”, the address generation unit 17acquires the learning parameter corresponding to the rectangular imageB1, and outputs the acquired learning parameter to the evaluation valuecalculation unit 14 in the classifier 7.

(3) Angle Set Unit 10

The angle set unit 10 switches between physical objects to be identifiedby the classifier 7. Specifically, the angle set unit 10 switchesbetween physical objects to be identified by switching between theselector tables used by the selector circuits 11 a and 11 b, and betweenthe address conversion tables used by the address generation unit 17.

The angle set unit 10 outputs an angle set signal to the rectangularimage cutout unit 6 and the ROM address calculation unit 8 both includedin the face detection unit 5. The angle set signal shows one of thestandard angle (0 degrees) and the rotating angles (90 degrees, reverse,and 270 degrees) of a face to be detected, that is, it shows a usableselector table and a usable address conversion table.

Angles set by the angle set unit 10 and angular ranges of a face to bedetected are shown in FIGS. 16A to 16D. Here, the angle indicates anangle measured by rotating a line which runs perpendicular to an inputimage in a counterclockwise direction. FIG. 16A shows an angular rangeat 0 degrees (standard angle). The face detection unit 5 performs facedetection in the range of ±α degrees (e.g. ±45 degrees) from anerected-state. Similarly, FIG. 16B shows an angular range at 90 degrees(rotating angle 90 degrees), FIG. 16C shows an angular range at 180degrees (rotating angle reverse), and FIG. 16D shows an angular range at270 degrees (rotating angle 270 degrees), respectively. Angular rangesin which the face detection is performed are the same as that shown inFIG. 16A.

Note that the standard angle (0 degrees) and the rotating angles (90degrees, reverse, and 270 degrees) are just examples, and an angle and atype of the angle are not limited to these. Also, when α showing theangular range in which the face detection is performed is 45 degrees,face detection from all rotating direction (360 degrees) can beperformed. However, it is not limited to 45 degrees.

The angle set signal is input to the rectangular image cutout unit 6 andthe ROM address calculation unit 8. In accordance with an angle shown bythe angle set signal, one of the selector tables can be used. Also, anaddress conversion table corresponding to the usable selector table canbe used. Specifically, in accordance with an angle shown by the angleset signal, (i) a selector table use flag corresponding to a selectortable to be used and (ii) an address conversion table use flagcorresponding to an address conversion table corresponding to theselector table are set to a value “1”, and (iii) selector table useflags corresponding to the other selector tables and (iv) addressconversion table use flags corresponding to the other address conversiontables are set to a value “0”. For example, when an angle shown by theangle set signal is a standard angle (0 degrees), a selector table useflag corresponding to the selector table T11 a is set to a value “1”,and selector table use flags corresponding to the other selector tablesare set to a value “0”. Also, an address conversion table use flagcorresponding to the address conversion table T31 a corresponding to theselector table T11 a is set to a value “1”, and address conversion tableuse flags corresponding to the other address conversion tables are setto “0”. In this way, the selector circuits 11 a, 11 b, and the addressgeneration unit 17 can specify the selector table T11 a which should beused at a standard angle (0 degrees), and the address conversion tableT31 a corresponding to the selector table T1 a.

A usable selector table and a usable address conversion table arerequired to be switched, because a cutout position of a rectangularimage is changed depending on an angle to be set. As shown in a specificexample, switching of a usable table can be realized by changing a valueof a flag corresponding to each table.

The angle set unit 10 receives, from a processing control unit 19,change information instructing to change an angle. Here, the changeinformation includes information showing an angle after being changed.

The angle set unit 10 outputs the angle set signal showing the angleincluded in the received change information to the rectangular imagecutout unit 6 and the ROM address calculation unit 8. Then, inaccordance with the angle, the angle set unit 10 allows one of theselector tables and one of the address conversion tables to be used.

For example, when information showing 0 degrees (standard angle 0degrees) is included in the change information, the angle set unit 10sets a value “1” to a selector table use flag and an address conversiontable use flag corresponding to the selector table T11 a and the addressconversion table T31 a, respectively, in accordance with the angle setsignal showing the standard angle 0 degrees. And, a value “0” is set toselector table use flags and address conversion table use flagscorresponding to the other selector tables and the other addressconversion tables, respectively.

(4) Others

The following describes a timing when the angle set unit 10 receives thechange information output from the processing control unit 19

When the image processing device 1 performs face detection processing,the processing control unit 19 outputs the change information to theangle set unit 10 such that the selector tables and the addressconversion tables are switched until a face is detected, or until facedetection processing for all angles (0 degrees, 90 degrees, reverse, and270 degrees) is performed. For example, at the start of the facedetection processing, the processing control unit 19 outputs the changeinformation showing the standard angle 0 degrees to the angle set unit10. When the classifier 7 judges that a face does not appear, theprocessing control unit 19 outputs the change information showing therotating angle 90 degrees to the angle set unit 10. When the classifier7 judges that a face appears, the processing control unit 19 inhibitsoutput of an angle set signal to the angle set unit 10. That is to say,the processing control unit 19 outputs the change information showingthe angle 0 degrees, 90 degrees, reverse, and 270 degrees, in thatorder, to the angle set unit 10 such that the selector tables and theaddress conversion tables are switched until the classifier 7 judgesthat a face appears. When a face is not detected for all angles, theprocessing control unit 19 inhibits face detection processing for awindow targeted for the detection.

In the above example, although an angle is switched from 0 degrees (atthe start of detection) to 90 degrees, reverse, and 270 degrees, in thisorder, until a face is detected in a window, the order of detection isnot limited to this. An angle at the start of detection may be any of 0degrees, 90 degrees, reverse, and 270 degrees. Also, as for the order ofswitching between angles, an angle may be switched by randomly selectingan angle corresponding to a table which is not permitted to be used.

1.2 Operation of the Image Processing Device 1

The following describes operation of the image processing device 1, andin particular, switching processing pertaining to switching betweenphysical objects to be identified, and face detection processing.

(1) Switching Processing

The following describes operation of switching processing with use of aflow chart shown in FIG. 17.

The angle set unit 10 receives switching information from the outside.(step S5)

When an angle included in the switching information is 0 degrees (“0degrees” in step S10), the angle set unit 10 enables only the selectortable T11 a in accordance with the angle set signal showing an angle 0degrees, and disables the other selector tables (step S15).Specifically, the angle set unit 10 sets a value “1” to a selector tableuse flag corresponding to the selector table T11 a in accordance withthe angle set signal showing an angle 0 degrees, and sets a value “0” toselector table use flags corresponding to the other selector tables.Also, the angle set unit 10 enables only the address conversion tableT31 a in accordance with the angle set signal showing an angle 0degrees, and disables the other address conversion tables (step S20).Specifically, the angle set unit 10 sets a value “1” to an addressconversion table use flag corresponding to the address conversion tableT31 a in accordance with the angle set signal showing an angle 0degrees, and sets a value “0” to address conversion table use flagscorresponding to the other address conversion tables.

When an angle included in the switching information is 90 degrees (“90degrees” in step S10), the angle set unit 10 enables only the selectortable T11 b in accordance with the angle set signal showing an angle 90degrees, and disables the other selector tables (step S25).Specifically, the angle set unit 10 sets a value “1” to a selector tableuse flag corresponding to the selector table T11 b in accordance withthe angle set signal showing an angle 90 degrees, and sets a value “0”to selector table use flags corresponding to the other selector tables.Also, the angle set unit 10 enables only the address conversion tableT31 b in accordance with the angle set signal showing an angle 90degrees, and disables the other address conversion tables (step S30).Specifically, the angle set unit 10 sets a value “1” to an addressconversion table use flag corresponding to the address conversion tableT31 b in accordance with the angle set signal showing an angle 90degrees, and sets a value “0” to address conversion table use flagscorresponding to the other address conversion tables.

When an angle included in the switching information is a reverse angle(180 degrees) (“reverse” in step S10), the angle set unit 10 enablesonly the selector table T11 c in accordance with the angle set signalshowing a reverse angle, and disables the other selector tables (stepS35). Specifically, the angle set unit 10 sets a value “1” to a selectortable use flag corresponding to the selector table T11 c in accordancewith the angle set signal showing a reverse angle, and sets a value “0”to selector table use flags corresponding to the other selector tables.Also, the angle set unit 10 enables only the address conversion tableT31 c in accordance with the angle set signal showing a reverse angle,and disables the other address conversion tables (step S40).Specifically, the angle set unit 10 sets a value “1” to an addressconversion table use flag corresponding to the address conversion tableT31 c in accordance with the angle set signal showing a reverse angle,and sets a value “0” to address conversion table use flags correspondingto the other address conversion tables.

When an angle included in the switching information is 270 degrees (“270degrees” in step S10), the angle set unit 10 enables only the selectortable T11 d in accordance with the angle set signal showing an angle 270degrees, and disables the other selector tables (step S45).Specifically, the angle set unit 10 sets a value “1” to a selector tableuse flag corresponding to the selector table T11 d in accordance withthe angle set signal showing an angle 270 degrees, and sets a value “0”to selector table use flags corresponding to the other selector tables.Also, the angle set unit 10 enables only the address conversion tableT31 d in accordance with the angle set signal showing an angle 270degrees, and disables the other address conversion tables (step S50).Specifically, the angle set unit 10 sets a value “1” to an addressconversion table use flag corresponding to the address conversion tableT31 d in accordance with the angle set signal showing an angle 270degrees, and sets a value “0” to address conversion table use flagscorresponding to the other address conversion tables.

(2) Operation of Face Detection Processing

The following describes operation of face detection processing with useof a flow chart shown in FIG. 18.

The selector circuits 11 a and 11 b cuts out one or more rectangularimages on lines in a window by using a usable selector table, i.e., aselector table with which a selector table use flag set to a value “1”is associated (step S100).

The characteristic amount calculation unit 13 calculates an amount ofcharacteristic for each of the one or more rectangular images cut out bythe selector circuits 11 a and 11 b (step S105).

The address generation unit 17 calculates, for each of the one or morerectangular images cut out by the selector circuits 11 a and 11 b, a ROMaddress (a reference address) of a learning parameter to be referredwith use of the amount of characteristic of the rectangular imagecalculated by the characteristic amount calculation unit 13 and a usableaddress conversion table (i.e. an address conversion table with which aaddress conversion table use flag set to a value “1” is associated)(step S110), and acquires the learning parameter corresponding to therectangular image based on the calculated ROM address (step S115).

The evaluation value calculation unit 14 calculates an evaluation valuefor the window with use of the leaning parameter acquired for each ofthe one or more rectangular images cut out by the selector circuits 11 aand 11 b (step S120).

The judgment unit 15 judges whether or not a face appears in the windowwith use of the calculated evaluation value (step S125), and outputs aresult of the judgment to the outside (step S130).

1.3 Modifications

Although the present invention has been explained in accordance with theabove embodiment 1, it is obvious that the present invention is notlimited to the above embodiment. The following modifications are alsoincluded in the present invention.

(1) The number of tiers the shift register has, a size and a shape of arectangular image, and the like shown in the above embodiment 1 areexamples, and not limited to these.

The number of tiers the shift register has needs to be only the same asa maximum vertical size of a rectangular image to be selected (to be cutout) or more.

Also, vertical pixels of a rectangular image may not be the same ashorizontal pixels of the rectangular image.

A rectangular image may have a plurality of shapes or size. In thiscase, shape information of the rectangular image (e.g. informationshowing a size of the rectangular image) is added to each column of therectangle information A and the rectangle information B in the selectortables shown in FIGS. 6 to 9. A shape selector for selecting a shape ofthe rectangular image is also added to the selector circuits 11 a and 11b in addition to the column selector circuit 22, and the shape selectorselects a rectangular image having a shape shown by the shapeinformation.

Also, a shape of an image to be selected is not limited to a rectangle.For example, an image to be selected may have a shape of a circle and anellipse. When a size of a circle to be selected is fixed, a coordinateshowing a center of the circle to be selected is given as the startposition. In contrast, when a size of a circle to be selected isvariable, a radius of the circle to be selected is additionally given asthe shape information.

(2) In the above embodiment 1, the selector circuit is described toselect (cut out) one rectangular image on one line, but it is notlimited to this.

One selector circuit may select (cut out) a plurality of rectangularimages on one line.

(3) In the above embodiment 1, the selector circuits 11 a and 11 b aredescribed to select (cut out) a rectangular image with reference to thesame selector table, but not limited to this.

Each selector circuit may be associated with different selector tables.

For example, selector tables composed of the window line number and therectangle information A shown in FIGS. 6 to 9 are associated with theselector circuit 11 a, and selector tables composed of the window linenumber and the rectangular image B are associated with the selectorcircuit 11 b.

(4) In the above embodiment 1, the image processing device 1 isdescribed to calculate the amount of characteristic after selecting arectangular image, but it is not limited to this. As shown in FIG. 19,the image processing device 1 a may preliminarily calculate the amountof characteristic of a rectangular image corresponding to all pixelslocations, and may select a result of the calculation. When manyrectangular images are included, since the calculation result of theamount of characteristic can be shared by preliminarily calculating theamount of characteristic, a calculation amount can be reduced as awhole.

(5) In the above embodiment 1, the image processing device 1 isdescribed to perform face detection processing as an example ofdetecting a physical object, but it is not limited to this.

A detection target is only required to have a characteristic part whichcan be identified as a physical object targeted for the detection (e.g.a face of an animal, an upper or whole body of a person).

(6) In the above embodiment 1, a flag is used to identify whether eachselector table and each address conversion table are enabled ordisabled, but it is not limited to this.

Any method to realize the identification may be adopted as long aswhether each table is enabled or disabled is identified by the method.For example, the selector table storage unit 12 has a referable area anda non-referable area, and four selector tables are stored in thenon-referable area. The angle set unit 10 copies only a usable selectortable to the referable area so that the selector table stored in thereferable area can be used. The above method may also be realized by theaddress conversion table storage unit having the referable area and thenon-referable area. That is to say, usable selector table and addressconversion table are stored in the referable areas in the selector tablestorage unit and the address conversion table storage unit,respectively. And the other selector tables and the other addressconversion tables are stored in the non-referable areas in the selectortable storage unit and the address conversion table storage unit,respectively. Here, the referable area is an area which is allowed to beaccessed from the outside (here, the selector circuits 11 a and 11 b,and the address generation unit 17), and the non-referable area is anarea which is not allowed to be accessed from the outside.

An access right showing whether or not each selector table and eachaddress conversion table can be accessed may be associated with each ofthe tables. In this case, the angle set unit 10 set the access rightsuch that only usable selector table and address conversion table areallowed to be accessed, and the other unusable selector tables andaddress conversion tables are not allowed to be accessed.

(7) In the above embodiment 1, an angle is described to be a standardangle 0 degrees, rotating angles 90 degrees and 270 degrees which arerotated in a counterclockwise direction from the standard angle, and areverse angle, but the angle is not limited to these.

The angle may be 0 degrees, 90 degrees, and 270 degrees, which arerotated in a clockwise direction from the standard angle, and a reverse.

In stead of the reverse angle, the angle may be a rotating angle 180degrees. In this case, a start position of the rectangular image afterbeing rotated 180 degrees has to be changed. For example, a startposition of the rectangular image B4 shown in FIG. 5A (x, y) is changedto (24-x-w, 24-y-h). It can be realized by changing the selector tableT11 c and the address conversion table T31 c based on a relationshipbetween a start position at 0 degrees (x, y) and a start position afterbeing rotated 180 degrees (24-x-w, 24-y-h). For example, a coordinate ofa start position of the rectangular image B1 at the rotating angle 180degrees becomes (24-x b-4, 24-y b-4).

(8) In the above embodiment 1, the processing control unit 19 isdescribed to be located outside of the image processing device 1, but itis not limited to this.

The processing control unit 19 may be located inside of the imageprocessing device 1.

(9) In the above embodiment 1, the selector table storage unit and theaddress conversion table storage unit are described to be separatelyconstructed, and store the selector tables 11 a to 11 d and addressconversion tables 31 a to 31 d, respectively, but it is not limited tothis.

One storage unit may store the selector tables 11 a to 11 d and theaddress conversion tables 31 a to 31 d.

Also, a selector table and a corresponding address conversion table maybe managed by one management table. In this case, the management tablehas, for example, an area for storing one or more combinations of thewindow line number, the rectangle information A, and the rectangleinformation B. The window line number is a value showing a line numberin a window. The rectangle information A and the rectangle information Bare composed of information about whether or not a rectangular image isincluded, an X-coordinate of the rectangular image, and an output (anoffset address). Since the information about whether or not arectangular image is included, the X-coordinate of the rectangularimage, and the output (the offset address) are described above,explanations thereof are omitted here.

(10) In the above embodiment 1, an angle is changed for a window until aface is detected, but it is not limited to this.

Face detection for image data (input image) to be shot may be performedby using one angle. When a face is not detected, face detection for theinput image may be performed by using another angle.

(11) The above embodiment and the above modifications may be combinedwith each other.

2. Embodiment 2

The following describes embodiment 2 of the present invention with useof FIG. 20.

Generally, a semiconductor integrated circuit 70 is composed of a MOStransistor like a CMOS, and a specific logic circuit is realized by aconnection structure of the MOS transistor. In resent years, since anintegration degree of a semiconductor integrated circuit has beenadvanced, a highly complex logic circuit (e.g. the image processingdevice in the present invention) can be realized by one or severalsemiconductor integrated circuits.

The semiconductor integrated circuit 70 includes the image processingdevice 1 described in embodiment 1, an image encoding circuit 71, anaudio processing unit 72, a ROM 73, a motion detection circuit 75, aprocessor 76, a control bus 77, a resolution conversion circuit 78, acamera input circuit 79, an LCD (Liquid Crystal Display) output circuit81, and an internal bus 83. These elements are connected with each othervia the internal bus 83, and input/output data.

Here, the image processing device 1 acquires an input image from theimage memory 74 via the internal bus 83.

The camera input circuit 79 receives a moving image from a camera 80,and performs processing to convert the received moving image to digitaldata.

The motion detection circuit 75 performs processing pertaining to motioncompensation for the moving image converted to digital data.

After the processing pertaining to the motion compensation is performedfor the moving image converted to digital data, the image encodingcircuit 71 performs processing pertaining to encoding of an image tostore the image in the image memory 74. Also, the image encoding circuit71 reads out the encoded image data from the image memory 74, anddecodes the read out encoded image data. The image processing device 1acquires the decoded image data.

The audio processing unit 72 performs processing to convert an audioinput from the camera 80 to audio data.

The ROM 73 has an area for storing data to be processed.

For example, the ROM 73 stores the audio data processed by the audioprocessing unit 72.

The resolution conversion circuit 78 performs processing to convertresolution of image data to be processed by the image processing device1.

The LCD output circuit 81 outputs image data to be displayed to the LCD(Liquid Crystal Display) 82. Also, as for an area in which a face isdetected, it causes the LCD 82 to display a frame surrounding the area.

The processor 76 controls processing pertaining to an entire operationof the semiconductor integrated circuit 70.

The processor 76 operates similarly to the processing control unit 19shown in the above embodiment 1, and controls switching between theselector tables and between the address conversion tables until theimage processing device 1 detects a face, or until face detectionprocessing for all angles is performed.

Also, the processor 76 controls, via the control bus 77, operation ofelements connected to the control bus 77.

In this embodiment, the angle set unit 10 is described to be included inthe image processing device 1 similarly to embodiment 1, but it is notlimited to this. The angle set unit 10 maybe included in the processor76. In other words, operation of the angle set unit 10 may be performedby the processor 76.

As described in embodiment 1, the image processing device 1 included inthe semiconductor integrated circuit 70 realizes face detectionprocessing in an image in which a face is rotated only by replacinginformation stored in tables. Accordingly, the image processing device 1can perform face detection processing in an image input from the camera80.

Also, since the image processing device 1 can be realized by thesemiconductor integrated circuit 70, a digital video camera and asurveillance camera can be downsized and power consumption can bereduced.

3. Embodiment 3

FIG. 21 is a block diagram showing embodiment 3 of the presentinvention.

An imaging device 100 includes a semiconductor integrated circuit 170, alens 89, a diaphragm 88, a sensor 87, an A/D conversion circuit 86, andan angle sensor 90.

The lens 89 is for imaging an object. Specifically, the lens 89 includesa zoom lens for realizing a zoom function and a focus lens for focusingon the object.

The diaphragm 88 includes a mechanical shutter, a diaphragm foradjusting brightness of the object, a filter, and the like. Thediaphragm 88 realizes functions of a shutter, diaphragm, and the like.

The sensor 87 includes a CCD (Charge Coupled Device), a CMOS(Complementary Metal Oxide Semiconductor), and the like. The sensor 87converts an object image imaged by the lens 89 into an electric signal,generates analog image data, and outputs the analog image data to theA/D conversion circuit 86.

The A/D conversion circuit 86 performs gain adjustment and A/D(analog/digital) conversion for analog data output from the sensor 87,and outputs image data as digital data.

The semiconductor integrated circuit 170 includes a zoom control 84 forcontrolling the lens 89 and an exposure control 85 for controlling thediaphragm 88 in addition to elements included in the semiconductorintegrated circuit 70 in embodiment 2.

The angle sensor 90 detects a shooting angle of the imaging device 100.The angle sensor 90 outputs, to the angle set unit 10, changeinformation showing an angle to be set based on the detected shootingangle.

Here, as shown in FIGS. 16A to 16D, when the detected shooting angle is±α degrees (e.g. ±45 degrees), the angle sensor 90 outputs an angle setsignal showing an angle 0 degrees to the angle set unit 10. When theshooting angle is within 45 to 135 degrees, or within −225 to −315degrees based on a standard angle 0 degrees, the angle sensor 90 outputsan angle set signal showing an angle 90 degrees to the angle set unit10. When the shooting angle is within 135 to 225 degrees, or within −135to −225 degrees based on the standard angle 0 degrees, the angle sensor90 outputs an angle set signal showing a reverse angle to the angle setunit 10. When the shooting angle is within 225 to 315 degrees, or within−45 to −135 degrees based on the standard angle 0 degrees, the anglesensor 90 outputs an angle set signal showing an angle 270 degrees tothe angle set unit 10.

Although the angle sensor 90 outputs the angle set signal to the angleset unit 10 here, but it is not limited to this. The angle sensor 90 mayonly detect the shooting angle of the imaging device 100, and theprocessor 76 may output the angle set signal to the angle set unit 10based on a detection result of the angle sensor 90.

By using positional information of a face detected by the imageprocessing device 1 included in the semiconductor integrated circuit170, the zoom control 84 can perform focus control and the exposurecontrol 85 can perform exposure control by adjusting a focus to a faceposition. Accordingly, the imaging device 100 which can shoot a faceclearly can be realized.

In this embodiment, the imaging device 100 is described to include theangle sensor 90, and switch the selector tables and the addressconversion tables to be used by the angle set unit 10 based on adetection result of the angle sensor 90, it is not limited to this. Forexample, the imaging device 100 may not include the angle sensor 90. Inthis case, as shown in embodiment 2, the processor 76 outputs changeinformation to the angle set unit 10 such that the selector tables andthe address conversion tables are switched from an angle 0 degrees to 90degrees, reverse, and 270 degrees, in this order, until the classifier 7judges that a face appears. Note that, as described in embodiment 2, theprocessor 76 inhibits face detection processing for a window targetedfor the detection when a face is not detected for all angles.

4. Modifications

Although the present invention has been explained in accordance with theabove embodiments 1 to 3 and the above modification, it is obvious thatthe present invention is not limited to the above embodiments andmodification. The following modifications are also included in thepresent invention.

(1) The processing control unit 19 in embodiment 1 and the processor 76in embodiment 2 are described to output change information to the angleset unit 10 such that the selector tables and the address conversiontables are switched until a face is detected, or until face detectionprocessing for all angles is performed, when the image processing device1 performs face detection processing, but it is not limited to this.Hereinafter, such operation is referred to as a first operation.

As shown in embodiment 3, the angle sensor, or the processing controlunit 19 (the processor 76) based on a detection result of the anglesensor may output change information to the angle set unit 10 such thatthe selector tables and the address conversion tables are switched.Hereinafter, such operation is referred to as a second operation.

Alternately, switching between tables using both of the first and secondoperation may be performed. In this case, a selector table and anaddress conversion table which is enabled based on a shooting angledetected by the angle sensor are used as tables to be used firstly inface detection, and selector tables and address conversion tables aresequentially switched until a face is detected. For example, theselector table and the address conversion table corresponding to anangle 90 degrees are used as tables to be used firstly in facedetection, selector tables and address conversion tables correspondingto an reverse angle, an angle 270 degrees, and 0 degrees aresequentially switched until a face is detected.

Also, switching between tables using both of the first and secondoperation may be performed in embodiment 3.

(2) The present invention may be realized by software.

(3) Part or all of elements constituting the above image processingdevice may include an IC card attachable/detachable to/from each deviceor a single module. The IC card and the module are computer systemscomposed of a microprocessor, a ROM, a RAM, and the like. The IC cardand the module may include the above ultra-multifunctional LSI. As themicroprocessor operates in accordance with a computer program, the ICcard and the module achieve its functions. The IC card and the modulemay be tamper resistant.

(4) Part or all of elements constituting the above image processingdevice may include one system LSI (Large Scale Integration). The systemLSI is an ultra-multifunctional LSI produced by integrating a pluralityof components on one chip. Specifically, the system LSI is a computersystem including a microprocessor, a ROM, a RAM, and the like. Acomputer program is stored in the RAM. As the microprocessor operates inaccordance with the computer program, the system LSI achieves itsfunction.

Each element constituting the above devices maybe achieved on one chip,or part or all thereof may be achieved on one chip.

Although a system LSI is taken for an example here, it may be called asIC, LSI, super LSI, and ultra LSI, depending on a difference inintegration degree. Also, the integrated circuit is not limited to theLSI, and it may be realized by a dedicated communication circuit or ageneral-purpose processor. It may be realized by FPGA (FieldProgrammable Gate Array) operable to program after production of a LSI,or a reconfigurable processor operable to reconfigure a connection or asetup of a circuit cell inside the LSI.

And further, if a technology of an integrated circuit replacing a LSIcomes out by a progress of a semiconductor technology or otherderivative technology thereof, naturally the technology may be used forintegration of functional blocks. Biotechnology and the like may beapplied to this kind of technology.

(5) The present invention maybe the above-mentioned method. Alternately,the present invention may be a computer program by which a computerachieves the above-mentioned method, a digital signal including thecomputer program.

Also, the present invention may be a computer-readable recording mediumlike a flexible disk, hard disc, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD(Blu-ray Disc), and semiconductor memory, recorded with the computerprogram or the digital signal. Alternately, the present invention may bethe computer program and the digital signal recorded on these recordingmedia.

Also, the present invention may be implemented by transmitting thecomputer program or the digital signal via a network represented by anelectric communication line, a wireless or a wired communication line,and the internet.

Also, the present invention may be a computer system including amicroprocessor and a memory. The above-mentioned computer program may bestored in the memory, and the microprocessor may operate in accordancewith the computer program.

Also, other independent computer system may implement the presentinvention by transferring the recording medium recorded with thecomputer program or the digital signal, or by transferring the computerprogram or the digital signal via the network and the like.

(6) The embodiments and the modification may be combined with eachother.

5. Summary

(1) One aspect of the present invention is an image processing devicecomprising: an image input unit from which an image is input, arectangular image cutout unit for cutting out a rectangular image fromthe image input from the image input unit, a classifier for (i)calculating an amount of characteristic for each of a plurality ofrectangular images cut out by the rectangular image cutout unit, (ii)calculating an evaluation value with use of the amount ofcharacteristic, and (iii) identifying whether a target object appearsbased on the calculated evaluation value, a learning parameter storingunit for storing a learning parameter used by the classifier tocalculate the evaluation value for each of the plurality of rectangularimages, a learning parameter reference calculation unit for showing areference of the learning parameter which is required to calculate theevaluation value for each of the plurality of rectangular images, and aswitching unit for switching cutout positions of each of the pluralityof rectangular images by changing (i) the cutout positions at which eachof the plurality of the rectangular images is cut out by the rectangularimage cutout unit and (ii) the reference of the learning parameter shownby the learning parameter reference calculation unit in accordance witha rotating angle to be set.

With the above structure, since the image processing device can set arelationship between the rectangular image and the learning parameterfor each rotating angle of an input image, face detection processing foreach rotated image can be performed without decreasing in face detectionspeed.

The image processing device can perform face detection processing foreach rotated image without decreasing in face detection speed in a facedetection method using a plurality of amounts of characteristic.

(2) Here, the rectangular image cutout unit may include a selector tablethat stores information about a cutout position of pixel data and aselector unit for selecting pixel data input from the image input unitin accordance with the selector table. The cutout position of therectangular image may be changed by changing the information stored inthe selector table.

(3) Here, the switching unit may switch the information about a cutoutposition of pixel data stored in the selector table.

(4) Here, the learning parameter reference calculation unit may includea learning parameter reference table for storing reference informationof the learning parameter which is required to calculate the evaluationvalue of the rectangular image. The reference information of thelearning parameter which is required to calculate the evaluation valueof the rectangular image may be changed by changing information storedin the learning parameter reference table.

(5) Here, the reference information of the learning parameter which isrequired to calculate the evaluation value of the rectangular imagestored in the learning parameter reference table may be switched by theswitching unit.

(6) Here, the switching unit may set an angle by using an angle of atarget object to be identified from an image input from the image inputunit.

(7) Here, the cutout position of the rectangular image switched by theswitching unit may correspond to a position shown by angle informationset by the switching unit based on a cutout position at 0 degrees.

(8) Here, the switching unit may set an angle as 90 degrees, 180degrees, or 270 degrees.

(9) Here, the learning parameter storing unit may be a ROM.

(10) Here, the reference information of the learning parameter may be anaddress of the ROM.

(11) Another aspect of the present invention is an image processingmethod comprising: inputting an image, cutting out a rectangular imagefrom the image input in the image input step, calculating an amount ofcharacteristic for each of a plurality of rectangular images cut out inthe rectangular image cutout step, calculating an evaluation value withuse of the amount of characteristic, and identifying whether a targetobject appears based on the calculated evaluation value, storing alearning parameter used in the identification step to calculate theevaluation value for each of the plurality of rectangular images,showing a reference of the learning parameter which is required tocalculate the evaluation value for each of the plurality of rectangularimages, and switching target objects to be identified in theidentification step by changing (i) the cutout positions at which eachof the plurality of the rectangular images is cut out in the rectangularimage cutout step and (ii) the reference of the learning parameter shownin the learning parameter reference calculation step.

(12) Another aspect of the present invention is a semiconductorintegrated circuit comprising: an image input unit from which an imageis input, a rectangular image cutout unit for cutting out a rectangularimage from the image input from the image input unit, a classifier for(i) calculating an amount of characteristic for each of a plurality ofrectangular images cut out by the rectangular image cutout unit, (ii)calculating an evaluation value with use of the amount ofcharacteristic, and (iii) identifying whether a target object appearsbased on the calculated evaluation value, a learning parameter storingunit for storing a learning parameter used by the classifier tocalculate the evaluation value for each of the plurality of rectangularimages, a learning parameter reference calculation unit for showing areference of the learning parameter which is required to calculate theevaluation value for each of the plurality of rectangular images, and aswitching unit for switching cutout positions of each of the pluralityof rectangular images by changing (i) the cutout positions at which eachof the plurality of the rectangular images is cut out by the rectangularimage cutout unit and (ii) the reference of the learning parameter shownby the learning parameter reference calculation unit.

(13) Yet another aspect of the present invention is an imaging devicecomprising: an image input unit from which an image is input, arectangular image cutout unit for cutting out a rectangular image fromthe image input from the image input unit, a classifier for (i)calculating an amount of characteristic for each of a plurality ofrectangular images cut out by the rectangular image cutout unit, (ii)calculating an evaluation value with use of the amount ofcharacteristic, and (iii) identifying whether a target object appearsbased on the calculated evaluation value, a learning parameter storingunit for storing a learning parameter used by the classifier tocalculate the evaluation value for each of the plurality of rectangularimages, a learning parameter reference calculation unit for showing areference of the learning parameter which is required to calculate theevaluation value for each of the plurality of rectangular images, and aswitching unit for switching cutout positions of each of the pluralityof rectangular images by changing (i) the cutout positions at which eachof the plurality of the rectangular images is cut out by the rectangularimage cutout unit and (ii) the reference of the learning parameter shownby the learning parameter reference calculation unit.

(14) In the present invention, since a relationship between a positionof a rectangular image to be selected and a rectangular area shown by atemplate frame is described to be variable, an image memory is accessedin the same manner as usual even in the case of a rotated image. Thiscauses, for example, when an image is rotated from FIG. 25A to FIG. 25B,a rectangular area in which matching processing is performed at the sametime when matching processing for a rectangular image R_B1 is performedis changed from a rectangular image R_A1 to a rectangular image R_A2. Inother words, a rectangular image to be accessed at the same time whenthe rectangular image R_B1 is accessed is changed from the rectangularimage R_A1 to the rectangular image R_A2.

In this way, the present invention can improve efficiency to access animage memory compared with conventional face detection processing forthe rotated image.

(15) Generally, since an image memory for storing an image has a largecapacity, the image memory is composed of a SRAM. The SRAM ischaracterized by being high in access efficiency when it is accessed inunits of word, and low in access efficiency when it is accessed in unitsof bit. This is because data of one word can be acquired by one access,whereas data of one column can be acquired by accessing the SRAM aplurality of times. Therefore, the image memory composed of the SRAM hascharacteristics relating to access efficiency which is the same as thatof the SRAM.

A large amount of throughput is required to perform face detectionprocessing. In order to complete processing within a predetermined time(e.g. 33 ms, when 30 frames are required to be processed in 1 minute),it is considered that matching processing performed by the facedetection unit is performed in parallel. FIGS. 25A and 25B show astorage state of an image which is not targeted for the detection intoan image memory. FIG. 25A shows a storage state in the case of a normalimage, whereas FIG. 25B shows a storage state in the case of a rotatedimage. R_A1 is a rectangular area for calculating an amount ofcharacteristic of the right eye, and R_B1 is a rectangular area forcalculating an amount of characteristic of the left eye. Also, avertical direction shows a direction in which words are arranged,whereas a horizontal direction shows a direction in which bits arearranged. For example, pixel data of the rectangular image R_A1 andpixel data of the rectangular image R_B1 in FIG. 25A can besimultaneously acquired by accessing an address (a1). Accordingly,matching processing for the rectangular area R_A1 and R_B1 can beperformed in parallel. However, in the case of the rotated image (FIG.25B), pixel data of the rectangular area R_A1 and pixel data of therectangular area R_B1 cannot be acquired at the same time . These piecesof pixel data are required to be acquired by sequentially accessing anaddress (b1) and an address (b2). Also, the matching processing can onlybe performed sequentially.

By having a selector table and an address conversion table for eachrotating angle (0 degrees, 90 degrees, reverse, and 270 degrees) asshown in embodiment 1, a rectangular image which should be selected whenthe selector circuit read out image of one line, and a parameter whichshould be used to evaluate the rectangular image can be properlyspecified. That is to say, face detection for a rotated image can beperformed in a face detection method using a plurality of amounts ofcharacteristic.

(16) Here, the image judgment device in the present inventioncorresponds to the image processing device 1 shown in the aboveembodiments.

The first storage unit in the present invention corresponds to the ROM 9in the above embodiments.

The second storage unit in the present invention corresponds to anelement composed of the selector table storage unit 12 and the addressconversion table storage unit 16 in the above embodiments.

The acquisition unit in the present invention corresponds to the imageinput unit 2 in the above embodiments.

The selection unit in the present invention corresponds to the angle setunit 10 in the above embodiments.

The extraction processing unit in the present invention corresponds toan element composed of the cutout processing unit 11 and characteristicamount calculation unit 13 in the above embodiments.

The judgment unit in the present invention corresponds to an elementcomposed of the evaluation value calculation unit 14, the judgment unit15, and the address generation unit 17 in the above embodiments.

The instruction unit in the present invention corresponds to theprocessing control unit 19 in the above embodiments.

The first placement position table and the second placement positiontable in the present invention correspond to the selector table in theabove embodiments.

The first existence position table and the second existence positiontable in the present invention correspond to the address conversiontable in the above embodiments.

INDUSTRIAL APPLICABILITY

The image processing device in the present invention performs facedetection processing for an input image in which a face is rotated, andis useful as a face detection device in a digital camera. Also, theimage processing device can be applied to a digital movie, asurveillance camera, and the like.

The present invention can be commercially, that is, repeatedly andcontinuously used in an industry in which an image processing device,and a digital video camera and digital camera including the imageprocessing device are manufactured and sold.

1-16. (canceled)
 17. An image judgment device for judging whether or nota characteristic part of an object appears in an image frame to be shot,the characteristic part being a distinguishing aspect of the object andhaving a plurality of specific elements, the judgment being made byusing the specific elements, the image judgment device comprising: afirst storage unit that stores element characteristic informationregarding an erected-state sample object, the element characteristicinformation showing characteristics of each of a plurality of specificelements that a characteristic part of the sample object has; a secondstorage unit that stores first positional information and secondpositional information, the first positional information defining aposition of each of a plurality of specific elements on a specificcoordinate system when a standard object is in the same state as theerected-state sample object, the second positional information defininga position of each of the specific elements on the specific coordinatesystem when the standard object has been rotated from the erected-state,the standard object having a characteristic part on which each of thespecific elements is standardly placed; an acquisition unit operable toacquire the image frame; a selection unit operable to select either thefirst positional information or the second positional information; anextraction processing unit operable to (i) apply the specific coordinatesystem to the image frame, (ii) when the first positional information isselected, acquire a partial image that is in the image frame andconsidered as an element specified by the first positional information,and acquire image characteristic information by extracting pixel valuesof the partial image in order in a direction along a first axis on thespecific coordinate system, and (iii) when the second positionalinformation is selected, acquire a partial image that is in the imageframe and considered as an element specified by the second positionalinformation, and acquire image characteristic information by extractingpixel values of the partial image in order in a direction along a secondaxis, the second axis being acquired by rotating the first axis in adirection the object is rotated, the order in the direction along thefirst axis being relatively the same as the order in the direction alongthe second axis; and a judgment unit operable to (i) specify the elementcharacteristic information for a specific element at a positioncorresponding to a position of the partial image on the specificcoordinate system, (ii) evaluate, for the specific element, the partialimage with use of the specified element characteristic information andthe acquired image characteristic information, and (iii) judge whetheror not the characteristic part having the specific element appears inthe image frame.
 18. The image judgment device in claim 17, wherein theextraction processing unit, (i) when the first axis is used, separates afirst partial image that is considered as the element specified by thefirst positional information into a plurality of areas in apredetermined separation method at a position based on the directionalong the first axis, defines one of the plurality of separated areas asa base area, and acquires the image characteristic information for thefirst partial image by sequentially scanning, starting from the basearea, the plurality of areas in a scanning order in the direction alongthe first axis, and (ii) when the second axis is used, separates asecond partial image that is considered as the element specified by thesecond positional information into a plurality of areas in thepredetermined separation method based on the direction along the secondaxis, and acquires the image characteristic information for the secondpartial image by sequentially scanning, starting from an area that isrelatively at the same position as the base area, the plurality of areasin a scanning order in the direction along the second axis, the scanningorder in the direction along the first axis being relatively the same asthe scanning order in the direction along the second axis.
 19. The imagejudgment device in claim 17, wherein the acquisition unit applies aframe surrounding the characteristic part of the standard object to theimage frame, and acquires a judgment image surrounded by the frame fromthe image frame, the extraction processing unit acquires a partial imagethat is in the judgment image and considered as an element specified byselected positional information, and acquires image characteristicinformation for the partial image, and the judgment unit (i) evaluates,for the specific element, the partial image with use of the specifiedelement characteristic information and the acquired image characteristicinformation, and (ii) judges whether or not the characteristic partappears in the judgment image.
 20. The image judgment device in claim 19further comprising: an instruction unit operable to, when the judgmentunit judges that the characteristic part does not appear in the judgmentimage, instruct the selection unit to select another piece of positionalinformation which has not been selected, wherein the selection unitselects the other piece of positional information which has not beenselected upon receiving the instruction, the extraction processing unitacquires another partial image that is considered as an elementspecified by the selected other piece of positional information, andacquires image characteristic information by extracting pixel values ofthe other partial image in order in a direction along an axiscorresponding to the selected other piece of positional information, andthe judgment unit (i) specifies the element characteristic informationfor a specific element at a position corresponding to a position of theother partial image on the specific coordinate system, (ii) evaluates,for the specific element, the other partial image with use of thespecified element characteristic information and the acquired imagecharacteristic information, and (iii) judges whether or not thecharacteristic part appears in the image frame.
 21. The image judgmentdevice in claim 19, wherein the extraction processing unit comprising:an image read sub-unit that has an area in which line images ofpredetermined lines in the judgment image are stored, and operable to(i) sequentially read a line image of a line from the judgment image,(ii) sequentially store the read line image in the area, and (iii)discard, from the area, a line image having been read first when thenumber of the stored line images exceeds the number of the predeterminedlines; and an extraction sub-unit operable to, each time the image readsub-unit reads and stores the line image, (i) acquire all partial imagesthat are in the line images stored in the image read sub-unit andconsidered as elements specified by selected positional information, and(ii) acquire image characteristic information for all the acquiredpartial images, and the judgment unit comprising: a specificationsub-unit operable to, each time the extraction sub-unit acquires thepartial image, specify element characteristic information for thepartial image; and a judgment sub-unit operable to judge whether or notthe characteristic part appears in the judgment image with use of theacquired image characteristic information and the specified elementcharacteristic information for all the acquired partial images.
 22. Theimage judgment device in claim 21, wherein the first positionalinformation is shown in a first placement position table indicating aplacement position of each of the specific elements that thecharacteristic part of the standard object has on the specificcoordinate system when the standard object is in the same state as theerected-state sample object, the second positional information is shownin a second placement position table indicating a placement position ofeach of the specific elements that the characteristic part of thestandard object has on the specific coordinate system when the standardobject has been rotated from the erected-state, the second storage unitfurther stores (i) a first existence position table in which theplacement position of each of the specific elements when the standardobject is in the same state as the erected-state sample object isassociated with an existence position of the element characteristicinformation for each of the specific elements in the first storage unit,and (ii) a second existence position table in which the placementposition of each of the specific elements when the object has beenrotated from the erected-state is associated with the existence positionof the element characteristic information for each of the specificelements, the selection unit selects either a pair of the firstplacement position table and the first existence position table or apair of the second placement position table and the second existenceposition table, the extraction sub-unit acquires the partial image withuse of a placement position table included in the pair selected by theselection unit, and the specification sub-unit specifies the elementcharacteristic information for the partial image with use of positionalinformation having been used to acquire the partial image and anexistence position table selected by the selection unit.
 23. The imagejudgment device in claim 22, wherein shape information showing a shapeof a partial image to be acquired is associated with the placementposition of each of the specific elements included in each of the firstplacement position table and the second placement position table, theextraction sub-unit acquires a partial image that is at the placementposition of each of the specific elements from the stored line images,the partial image having a shape shown by the shape informationcorresponding to the placement position.
 24. The image judgment devicein claim 22, wherein the element characteristic information is anevaluation value group composed of evaluation values having beenweighted in terms of an approximation to a corresponding specificelement, each amount of characteristic shown by a combination ofbrightness and darkness in a plurality of areas constituting the partialimage is associated with a different sample value, the existenceposition included in the existence position table shows a startingposition of a corresponding sample value group in the first storageunit, the extraction sub-unit acquires, as the image characteristicinformation, the amount of characteristic of the partial image byextracting pixel values of the partial image in order in a directionalong an axis corresponding to the selected positional information, thespecification sub-unit specifies, each time the extraction sub-unitacquires the partial image, a starting position of the evaluation valuegroup with use of the selected existence position table and theplacement position having been used to acquire the partial image, thejudgment sub-unit (i) acquires an evaluation value corresponding to theamount of characteristic of the partial image from the evaluation valuegroup shown by the specified starting position, (ii) calculates aweighted value in terms of an approximation to the characteristic partfrom a total of the acquired evaluation values for all the partialimages in the judgment image, and (iii) judges that the characteristicpart appears in the judgment image when the calculated weighted value isa predetermined threshold value or more.
 25. The image judgment devicein claim 22, wherein the second storage unit further stores, (i) inassociation with the first placement position table and the firstexistence position table, a first placement position table use flag anda first existence position table use flag, respectively, showing whetheror not each of the tables can be used and, (ii) in association with thesecond placement position table and the second existence position table,a second placement position table use flag and a second existenceposition table use flag, respectively, showing whether or not each ofthe tables can be used, the selection unit further sets a value for aplacement position table use flag and an existence position table useflag corresponding to a pair of the placement position table and theexistence position table to be selected, the value showing that thecorresponding placement position table and existence position table arepermitted to be used, the extraction sub-unit uses the placementposition table for which the value showing that the placement positiontable is permitted to be used is set to acquire the partial image, andthe specification sub-unit uses the existence position table for whichthe value showing that the existence position table is permitted to beused is set to specify the element characteristic information.
 26. Theimage judgment device in claim 22, wherein the second storage unit has areferable area and a non-referable area, the selection unit stores apair of the placement position table and the existence position table tobe selected in the referable area, and the other pairs of the placementposition table and the existence position table in the non-referablearea, the extraction sub-unit uses the placement position table storedin the referable area to acquire the partial image, and thespecification sub-unit uses the existence position table stored in thereferable area to specify the element characteristic information. 27.The image judgment device in claim 22, wherein an access right isassociated with each of the first placement position table, the secondplacement position table, the first existence position table, and thesecond existence position table, the access right showing whether or noteach of the tables can be accessed, the selection unit sets (i)an accessright for a pair of the placement position table and the existenceposition table to be selected, the access right showing that thecorresponding placement position table and existence position table arepermitted to be accessed, and (ii) an access right for pairs of theother placement position tables and existence position tables, theaccess right showing that the corresponding placement position tablesand existence position tables are not permitted to be accessed, theextraction sub-unit uses the placement position table for which theaccess right showing the placement position table is permitted to beaccessed is set to acquire the partial image, and the specificationsub-unit uses the existence position table for which the access rightshowing the existence position table is permitted to be accessed is setto specify the element characteristic information.
 28. The imagejudgment device in claim 17, wherein the selection unit receives aninstruction to select either the first or second positional informationfrom the outside, and selects positional information shown by thereceived instruction.
 29. The image judgment device in claim 17, whereinthe object is rotated 90 degrees, 180 degrees, 270 degrees, or reversed.30. The image judgment device in claim 17, wherein the object is aperson, and the characteristic part is a face.
 31. The image judgmentdevice in claim 17 is included in an imaging device shooting the object.32. An image judgment method that is used in an image judgment devicefor judging whether or not a characteristic part of an object appears inan image frame to be shot, the characteristic part being adistinguishing aspect of the object and having a plurality of specificelements, the judgment being made by using the specific elements, theimage judgment device including: a first storage unit that storeselement characteristic information regarding an erected-state sampleobject, the element characteristic information showing characteristicsof each of a plurality of specific elements that a characteristic partof the sample object has; a second storage unit that stores firstpositional information and second positional information, the firstpositional information defining a position of each of a plurality ofspecific elements on a specific coordinate system when a standard objectis in the same state as the erected-state sample object, the secondpositional information defining a position of each of the specificelements on the specific coordinate system when the standard object hasbeen rotated from the erected-state, the standard object having acharacteristic part on which each of the specific elements is standardlyplaced, and the image judgment method comprising the steps of: acquiringthe image frame; selecting either the first positional information orthe second positional information; (i) applying the specific coordinatesystem to the image frame, (ii) when the first positional information isselected, acquiring a partial image that is in the image frame andconsidered as an element specified by the first positional information,and acquiring image characteristic information by extracting pixelvalues of the partial image in order in a direction along a first axison the specific coordinate system, and (iii) when the second positionalinformation is selected, acquiring a partial image that is in the imageframe and considered as an element specified by the second positionalinformation, and acquiring image characteristic information byextracting pixel values of the partial image in order in a directionalong a second axis, the second axis being acquired by rotating thefirst axis in a direction the object is rotated, the order in thedirection along the first axis being relatively the same as the order inthe direction along the second axis; and (i) specifying the elementcharacteristic information for a specific element at a positioncorresponding to a position of the partial image on the specificcoordinate system, (ii) evaluating, for the specific element, thepartial image with use of the specified element characteristicinformation and the acquired image characteristic information, and (iii)judging whether or not the characteristic part having the specificelement appears in the image frame.
 33. An integrated circuit that isused in an image judgment device for judging whether or not acharacteristic part of an object appears in an image frame to be shot,the characteristic part being a distinguishing aspect of the object andhaving a plurality of specific elements, the judgment being made byusing the specific elements, the integrated circuit comprising: a firststorage unit that stores element characteristic information regarding anerected-state sample object, the element characteristic informationshowing characteristics of each of a plurality of specific elements thata characteristic part of the sample object has; a second storage unitthat stores first positional information and second positionalinformation, the first positional information defining a position ofeach of a plurality of specific elements on a specific coordinate systemwhen a standard object is in the same state as the erected-state sampleobject, the second positional information defining a position of each ofthe specific elements on the specific coordinate system when thestandard object has been rotated from the erected-state, the standardobject having a characteristic part on which each of the specificelements is standardly placed; an acquisition unit operable to acquirethe image frame; a selection unit operable to select either the firstpositional information or the second positional information; anextraction processing unit operable to (i) apply the specific coordinatesystem to the image frame, (ii) when the first positional information isselected, acquire a partial image that is in the image frame andconsidered as an element specified by the first positional information,and acquire image characteristic information by extracting pixel valuesof the partial image in order in a direction along a first axis on thespecific coordinate system, and (iii) when the second positionalinformation is selected, acquire a partial image that is in the imageframe and considered as an element specified by the second positionalinformation, and acquire image characteristic information by extractingpixel values of the partial image in order in a direction along a secondaxis, the second axis being acquired by rotating the first axis in adirection the object is rotated, the order in the direction along thefirst axis being relatively the same as the order in the direction alongthe second axis; and a judgment unit operable to (i) specify the elementcharacteristic information for a specific element at a positioncorresponding to a position of the partial image on the specificcoordinate system, (ii) evaluate, for the specific element, the partialimage with use of the specified element characteristic information andthe acquired image characteristic information, and (iii) judge whetheror not the characteristic part having the specific element appears inthe image frame.